fchinembiri
/
geocrop-platform.
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
geocrop-platform./plan/02_dynamic_tiler.md

452 lines
10 KiB
Markdown
Raw Blame History

# Plan 02: Dynamic Tiler Service (TiTiler)
**Status**: Pending Implementation
**Date**: 2026-02-27
---
## Objective
Deploy a dynamic tiling service to serve Cloud Optimized GeoTIFFs (COGs) from MinIO as XYZ map tiles for the React frontend. This enables efficient map rendering without downloading entire raster files.
---
## 1. Architecture Overview
```mermaid
graph TD
A[React Frontend] -->|Tile Request XYZ/zoom/x/y| B[Ingress]
B --> C[TiTiler Service]
C -->|Read COG tiles| D[MinIO]
C -->|Return PNG/Tiles| A
E[Worker] -->|Upload COG| D
F[API] -->|Generate URLs| C
```
---
## 2. Technology Choice
### 2.1 TiTiler vs Rio-Tiler
| Feature | TiTiler | Rio-Tiler |
|---------|---------|-----------|
| Deployment | Docker/Cloud Native | Python Library |
| API REST | ✅ Built-in | ❌ Manual |
| Cloud Optimized | ✅ Native | ✅ Native |
| Multi-source | ✅ Yes | ✅ Yes |
| Dynamic tiling | ✅ Yes | ✅ Yes |
| **Recommendation** | **TiTiler** | - |
**Chosen**: **TiTiler** (modern, API-first, Kubernetes-ready)
### 2.2 Alternative: Custom Tiler with Rio-Tiler
If TiTiler has issues, implement custom FastAPI endpoint:
- Use `rio-tiler` as library
- Create `/tiles/{job_id}/{z}/{x}/{y}` endpoint
- Read from MinIO on-demand
---
## 3. Deployment Strategy
### 3.1 Kubernetes Deployment
Create `k8s/25-tiler.yaml`:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: geocrop-tiler
namespace: geocrop
labels:
app: geocrop-tiler
spec:
replicas: 2
selector:
matchLabels:
app: geocrop-tiler
template:
metadata:
labels:
app: geocrop-tiler
spec:
containers:
- name: tiler
image: ghcr.io/developmentseed/titiler:latest
ports:
- containerPort: 8000
env:
- name: MINIO_ENDPOINT
value: "minio.geocrop.svc.cluster.local:9000"
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: geocrop-secrets
key: minio-access-key
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: geocrop-secrets
key: minio-secret-key
- name: AWS_S3_ENDPOINT_URL
value: "http://minio.geocrop.svc.cluster.local:9000"
- name: TILED_READER
value: "cog"
resources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "2Gi"
cpu: "1000m"
livenessProbe:
httpGet:
path: /healthz
port: 8000
initialDelaySeconds: 10
periodSeconds: 30
readinessProbe:
httpGet:
path: /healthz
port: 8000
initialDelaySeconds: 5
periodSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
name: geocrop-tiler
namespace: geocrop
spec:
selector:
app: geocrop-tiler
ports:
- port: 8000
targetPort: 8000
type: ClusterIP
```
### 3.2 Ingress Configuration
Add to existing ingress or create new:
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: geocrop-tiler
namespace: geocrop
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: "50m"
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
ingressClassName: nginx
tls:
- hosts:
- tiles.portfolio.techarvest.co.zw
secretName: geocrop-tiler-tls
rules:
- host: tiles.portfolio.techarvest.co.zw
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: geocrop-tiler
port:
number: 8000
```
### 3.3 DNS Configuration
Add A record:
- `tiles.portfolio.techarvest.co.zw` → `167.86.68.48` (ingress IP)
---
## 4. TiTiler API Usage
### 4.1 Available Endpoints
| Endpoint | Description |
|----------|-------------|
| `GET /cog/tiles/{z}/{x}/{y}.png` | Get tile as PNG |
| `GET /cog/tiles/{z}/{x}/{y}.webp` | Get tile as WebP |
| `GET /cog/point/{lon},{lat}` | Get pixel value at point |
| `GET /cog/bounds` | Get raster bounds |
| `GET /cog/info` | Get raster metadata |
| `GET /cog/stats` | Get raster statistics |
### 4.2 Tile URL Format
```javascript
// For a COG in MinIO:
const tileUrl = `https://tiles.portfolio.techarvest.co.zw/cog/tiles/{z}/{x}/{y}.png?url=s3://geocrop-results/jobs/${jobId}/refined.tif`;
// Or with custom colormap:
const tileUrl = `https://tiles.portfolio.techarvest.co.zw/cog/tiles/{z}/{x}/{y}.png?url=s3://geocrop-results/jobs/${jobId}/refined.tif&colormap=${colormapId}`;
```
### 4.3 Multiple Layers
```javascript
// True color (Sentinel-2)
const trueColorUrl = `https://tiles.portfolio.techarvest.co.zw/cog/tiles/{z}/{x}/{y}.png?url=s3://geocrop-results/jobs/${jobId}/truecolor.tif`;
// NDVI
const ndviUrl = `https://tiles.portfolio.techarvest.co.zw/cog/tiles/{z}/{x}/{y}.png?url=s3://geocrop-results/jobs/${jobId}/ndvi_peak.tif&colormap=ndvi`;
// DW Baseline
const dwUrl = `https://tiles.portfolio.techarvest.co.zw/cog/tiles/{z}/{x}/{y}.png?url=s3://geocrop-baselines/DW_Zim_HighestConf_${year}/${year+1}.tif`;
```
---
## 5. Color Mapping
### 5.1 Crop Classification Colors
Define colormap for LULC classes:
```json
{
"colormap": {
"0": [27, 158, 119], // cropland - green
"1": [229, 245, 224], // forest - dark green
"2": [247, 252, 245], // grass - light green
"3": [224, 236, 244], // shrubland - teal
"4": [158, 188, 218], // water - blue
"5": [240, 240, 240], // builtup - gray
"6": [150, 150, 150], // bare - brown/gray
}
}
```
### 5.2 NDVI Color Scale
Use built-in `viridis` or custom:
```javascript
const ndviColormap = {
0: [68, 1, 84], // Low - purple
100: [253, 231, 37], // High - yellow
};
```
---
## 6. Frontend Integration
### 6.1 React Leaflet Integration
```javascript
// Using react-leaflet
import { TileLayer } from 'react-leaflet';
// Main result layer
<TileLayer
url={`https://tiles.portfolio.techarvest.co.zw/cog/tiles/{z}/{x}/{y}.png?url=s3://geocrop-results/jobs/${jobId}/refined.tif`}
attribution='&copy; GeoCrop'
/>
// DW baseline comparison
<TileLayer
url={`https://tiles.portfolio.techarvest.co.zw/cog/tiles/{z}/{x}/{y}.png?url=s3://geocrop-baselines/DW_Zim_HighestConf_${year}/${year+1}.tif`}
attribution='Dynamic World'
/>
```
### 6.2 Layer Switching
Implement layer switcher in React:
```javascript
const layerOptions = [
{ id: 'refined', label: 'Refined Crop Map', urlTemplate: '...' },
{ id: 'dw', label: 'Dynamic World Baseline', urlTemplate: '...' },
{ id: 'truecolor', label: 'True Color', urlTemplate: '...' },
{ id: 'ndvi', label: 'Peak NDVI', urlTemplate: '...' },
];
```
---
## 7. Performance Optimization
### 7.1 Caching Strategy
TiTiler automatically handles tile caching, but add:
```yaml
# Kubernetes annotations for caching
annotations:
nginx.ingress.kubernetes.io/enable-access-log: "false"
nginx.ingress.kubernetes.io/proxy-cache-valid: "200 1h"
```
### 7.2 MinIO Performance
- Ensure COGs have internal tiling (256x256)
- Use DEFLATE compression
- Set appropriate overview levels
### 7.3 TiTiler Configuration
```python
# titiler/settings.py
READER = "cog"
CACHE_CONTROL = "public, max-age=3600"
TILES_CACHE_MAX_AGE = 3600 # seconds
# Environment variables for S3/MinIO
AWS_ACCESS_KEY_ID=minioadmin
AWS_SECRET_ACCESS_KEY=minioadmin12
AWS_REGION=dummy
AWS_S3_ENDPOINT=http://minio.geocrop.svc.cluster.local:9000
AWS_HTTPS=NO
```
---
## 8. Security
### 8.1 MinIO Access
TiTiler needs read access to MinIO:
- Use IAM-like policies via MinIO
- Restrict to specific buckets
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {"AWS": ["arn:aws:iam::system:user/tiler"]},
"Action": ["s3:GetObject"],
"Resource": [
"arn:aws:s3:::geocrop-results/*",
"arn:aws:s3:::geocrop-baselines/*"
]
}
]
}
```
### 8.2 Ingress Security
- Keep TLS enabled
- Consider rate limiting on tile endpoints
### 8.3 Security Model (Portfolio-Safe)
Two patterns:
**Pattern A (Recommended): API Generates Signed Tile URLs**
- Frontend requests "tile access token" per job layer
- API issues short-lived signed URL(s)
- Frontend uses those URLs as tile template
**Pattern B: Tiler Behind Auth Proxy**
- API acts as proxy adding Authorization header
- More complex
Start with Pattern A if TiTiler can read signed URLs; otherwise Pattern B.
---
## 9. Implementation Checklist
- [ ] Create Kubernetes deployment manifest for TiTiler
- [ ] Create Service
- [ ] Create Ingress with TLS
- [ ] Add DNS A record for tiles subdomain
- [ ] Configure MinIO bucket policies for TiTiler access
- [ ] Deploy to cluster
- [ ] Test tile endpoint with sample COG
- [ ] Verify performance (< 1s per tile)
- [ ] Integrate with frontend
---
## 10. Alternative: Custom Tiler Service
If TiTiler has compatibility issues, implement custom:
```python
# apps/tiler/main.py
from fastapi import FastAPI, HTTPException
from rio_tiler.io import COGReader
import boto3
app = FastAPI()
s3 = boto3.client('s3',
endpoint_url='http://minio.geocrop.svc.cluster.local:9000',
aws_access_key_id=os.getenv('AWS_ACCESS_KEY_ID'),
aws_secret_access_key=os.getenv('AWS_SECRET_ACCESS_KEY'),
)
@app.get("/tiles/{job_id}/{z}/{x}/{y}.png")
async def get_tile(job_id: str, z: int, x: int, y: int):
s3_key = f"jobs/{job_id}/refined.tif"
# Generate presigned URL (short expiry)
presigned_url = s3.generate_presigned_url(
'get_object',
Params={'Bucket': 'geocrop-results', 'Key': s3_key},
ExpiresIn=300
)
# Read tile with rio-tiler
with COGReader(presigned_url) as cog:
tile = cog.tile(x, y, z)
return Response(tile, media_type="image/png")
```
---
## 11. Technical Notes
### 11.1 COG Requirements
For efficient tiling, COGs must have:
- Internal tiling (256x256)
- Overviews at multiple zoom levels
- Appropriate compression
### 11.2 Coordinate Reference System
Zimbabwe uses:
- EPSG:32736 (UTM Zone 36S) for local
- EPSG:4326 (WGS84) for web tiles
TiTiler handles reprojection automatically.
### 11.3 Tile URL Expiry
For signed URLs:
- Generate with long expiry (24h) for job results
- Or use bucket policies for public read
- Pass URL as query param to TiTiler
---
## 12. Next Steps
After implementation approval:
1. Create TiTiler Kubernetes manifests
2. Configure ingress and TLS
3. Set up DNS
4. Deploy and test
5. Integrate with frontend layer switcher
Reference in New Issue View Git Blame Copy Permalink