When archaeologists analyze LiDAR point clouds, they see terrain elevation changes — subtle mounds, ditches, and depressions that might indicate ancient structures. But LiDAR alone doesn’t tell the full story. What was the environment like? What grew there? How did the landscape change over time?
This is where satellite imagery becomes crucial. In Archaios, after processing LiDAR data to detect terrain anomalies, we automatically fetch satellite imagery from Google Earth Engine to analyze:
🌾 Vegetation Patterns (NDVI)
- Crops grow differently over buried walls (shallow roots = stressed plants)
- Ancient ditches retain more moisture (healthier, darker green vegetation)
- Roman roads with compacted soil show vegetation gaps
- NDVI highlights these differences even when invisible to the eye
🌍 Visual Context (True Color)
- Natural RGB view showing modern field boundaries
- Orientation and landscape context for archaeologists
- Validation against known features
🔴 Subsurface Indicators (False Color Infrared)
- Healthy vegetation appears bright red
- Disturbed soil from ancient construction appears blue-green
- Archaeological features stand out as red vegetation gaps
1
2
3
4
5
6
7
8
9
10
11
12
|
User uploads LiDAR file
↓
Extract coordinates from point cloud
↓
[Parallel Processing]
├─> Store LiDAR DTM/DSM/Hillshade
└─> Fetch satellite imagery from Google Earth Engine
├─> NDVI (vegetation stress)
├─> True Color (visual context)
└─> False Color (infrared analysis)
↓
Feed all imagery to AI agents for archaeological assessment
|
Google Earth Engine gives us access to petabytes of Landsat and Sentinel-2 data going back decades — perfect for analyzing seasonal vegetation changes over archaeological sites.
I built a Python FastAPI microservice that runs as an Azure Container App. When the main Durable Functions orchestrator processes a LiDAR file, it extracts the site coordinates and calls this service via HTTP to fetch three types of satellite imagery in parallel.
Why separate service?
- Earth Engine Python SDK requires specific dependencies (geemap, rasterio, GDAL)
- Processing can take 30-60 seconds per image (Earth Engine API calls)
- Keeps the main .NET orchestrator lightweight
- Allows independent scaling of satellite processing
Why FastAPI?
- Async endpoints handle Earth Engine’s slow API calls efficiently
- Clean REST API for .NET activities to consume
- Built-in request validation with Pydantic models
- Easy to add new imagery types (e.g., moisture index, thermal)
The satellite imagery processing runs as a sub-orchestration that executes in parallel with LiDAR data storage:
1
2
3
4
5
6
7
8
9
10
|
[Main Orchestrator]
├─> InstantiateLiDARDataNode (store LiDAR results)
└─> GeeProcessingSubOrchestration (parallel)
├─> ProcessNdviImagery
├─> ProcessTrueColorImagery
└─> ProcessFalseColorImagery
↓ (all call)
[FastAPI GEE Processor]
↓
[Google Earth Engine API]
|
Here’s the actual sub-orchestrator that coordinates NDVI, true color, and false color processing:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
|
[Function(nameof(GeeProcessingSubOrchestration))]
public async Task<SatelliteImageryResult?> RunGeeProcessingSubOrchestration([OrchestrationTrigger] TaskOrchestrationContext context)
{
var geeMessage = context.GetInput<GeeCoordinateMessage>();
if (geeMessage == null || geeMessage.Coordinates == null)
{
await context.CallActivityAsync("LogGeeProcessingError",
new GeeProcessingError { SiteId = "unknown", Error = "Invalid GEE message received in orchestration" });
return default;
}
try
{
// Launch all three imagery types in parallel
var ndviTask = context.CallActivityAsync<GeeImageResult>(nameof(ProcessNdviImagery), geeMessage);
var trueColorTask = context.CallActivityAsync<GeeImageResult>(nameof(ProcessTrueColorImagery), geeMessage);
var falseColorTask = context.CallActivityAsync<GeeImageResult>(nameof(ProcessFalseColorImagery), geeMessage);
// Wait for all three to complete
await Task.WhenAll(ndviTask, trueColorTask, falseColorTask);
var satelliteData = new SatelliteImageryResult
{
SiteId = geeMessage.SiteId,
NdviImageUrl = ndviTask.Result?.ImageUrl,
TrueColorImageUrl = trueColorTask.Result?.ImageUrl,
FalseColorImageUrl = falseColorTask.Result?.ImageUrl,
ProcessedAt = DateTime.UtcNow
};
// Update Neo4j with satellite imagery URLs
await context.CallActivityAsync(nameof(UpdateSiteWithImagery),
new SiteImageryUpdateRequest {
SiteId = geeMessage.SiteId,
ImageryData = satelliteData,
Latitude = geeMessage.Coordinates.Latitude,
Longitude = geeMessage.Coordinates.Longitude
});
_logger.LogInformation($"Successfully processed GEE data for site {geeMessage.SiteId}");
return satelliteData;
}
catch (Exception ex)
{
await context.CallActivityAsync(nameof(LogGeeProcessingError),
new GeeProcessingError { SiteId = geeMessage.SiteId, Error = ex.Message });
return default;
}
}
|
Why this pattern?
Each imagery type takes 30-60 seconds to process (Earth Engine API calls are slow). Running them sequentially would take 2-3 minutes. With Task.WhenAll, they complete in ~60 seconds total.
If one image type fails (e.g., no cloud-free imagery available), the orchestration continues — the AI agents can still analyze partial satellite data.
Each activity function makes an HTTP POST to the Python FastAPI service running in a Container App:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
|
public class ProcessNdviImagery
{
private readonly ILogger<ProcessNdviImagery> _logger;
private readonly HttpClient _httpClient;
private readonly IConfiguration _configuration;
public ProcessNdviImagery(
ILogger<ProcessNdviImagery> logger,
HttpClient httpClient,
IConfiguration configuration)
{
_logger = logger;
_httpClient = httpClient;
_configuration = configuration;
}
[Function("ProcessNdviImagery")]
public async Task<GeeImageResult> Run([ActivityTrigger] GeeCoordinateMessage message)
{
_logger.LogInformation($"Processing NDVI imagery for site {message.SiteId}");
try
{
message.AnalysisType = "ndvi";
string geeProcessorUrl = _configuration.GetValue<string>("GeeProcessor:Endpoint") + "/process/ndvi";
var content = new StringContent(
JsonConvert.SerializeObject(message),
Encoding.UTF8,
"application/json"
);
var response = await _httpClient.PostAsync(geeProcessorUrl, content);
if (!response.IsSuccessStatusCode)
{
var errorContent = await response.Content.ReadAsStringAsync();
_logger.LogError($"GEE Processor error ({response.StatusCode}): {errorContent}");
return null;
}
var resultJson = await response.Content.ReadAsStringAsync();
var result = JsonConvert.DeserializeObject<GeeImageResult>(resultJson);
return result ?? new GeeImageResult
{
ImageType = "NDVI",
ImageUrl = null,
Collection = message.Collection.Split('/').LastOrDefault() ?? "Landsat",
ProcessedDate = DateTime.UtcNow,
Description = "Normalized Difference Vegetation Index"
};
}
catch (Exception ex)
{
_logger.LogError(ex, $"Error processing NDVI imagery for site {message.SiteId}");
return null;
}
}
}
|
The GeeProcessor:Endpoint configuration points to the Container App URL (e.g., https://archaios-gee.centralindia-01.azurecontainerapps.io).
This pattern repeats for ProcessTrueColorImagery and ProcessFalseColorImagery — same HTTP call, different endpoint (/process/truecolor, /process/falsecolor).
The Python service exposes three endpoints — one for each imagery type:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
|
from fastapi import APIRouter, HTTPException, Depends
from core.models import GeeCoordinateMessage, GeeImageResult
from services.gee_service import GeeService
from processors.gee_processor import GeeProcessor
from infrastructure.callback_service import CallbackService
import logging
router = APIRouter()
logger = logging.getLogger("Archaios.API")
def get_gee_service():
"""Get the GEE service instance."""
processor = GeeProcessor()
callback_service = CallbackService()
return GeeService(processor, callback_service)
@router.get("/health")
async def health_check():
"""Health check endpoint."""
logger.info("Health check endpoint called")
return {"status": "healthy"}
@router.post("/process/ndvi", response_model=GeeImageResult)
async def process_ndvi(message: GeeCoordinateMessage, service: GeeService = Depends(get_gee_service)):
"""Process NDVI imagery for the provided coordinates."""
logger.info(f"Received NDVI processing request for site {message.siteId}")
if not message.coordinates.has_valid_coordinates:
logger.warning(f"Invalid coordinates (0,0) for site {message.siteId}")
raise HTTPException(status_code=400, detail="Invalid coordinates")
result = await service.process_ndvi(message)
return result
@router.post("/process/truecolor", response_model=GeeImageResult)
async def process_true_color(message: GeeCoordinateMessage, service: GeeService = Depends(get_gee_service)):
"""Process true color imagery for the provided coordinates."""
logger.info(f"Received true color processing request for site {message.siteId}")
if not message.coordinates.has_valid_coordinates:
logger.warning(f"Invalid coordinates (0,0) for site {message.siteId}")
raise HTTPException(status_code=400, detail="Invalid coordinates")
result = await service.process_true_color(message)
return result
@router.post("/process/falsecolor", response_model=GeeImageResult)
async def process_false_color(message: GeeCoordinateMessage, service: GeeService = Depends(get_gee_service)):
"""Process false color imagery for the provided coordinates."""
logger.info(f"Received false color processing request for site {message.siteId}")
if not message.coordinates.has_valid_coordinates:
logger.warning(f"Invalid coordinates (0,0) for site {message.siteId}")
raise HTTPException(status_code=400, detail="Invalid coordinates")
result = await service.process_false_color(message)
return result
|
Each endpoint validates coordinates, calls the GeeService, and returns a GeeImageResult with the blob storage URL.
NDVI (Normalized Difference Vegetation Index) is calculated as (NIR - Red) / (NIR + Red). Healthy vegetation reflects a lot of near-infrared light but absorbs red light, giving high NDVI values (0.6-0.9). Stressed vegetation has lower NDVI (0.2-0.4).
Here’s how we calculate it using Google Earth Engine:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
|
async def process_ndvi(self, message: GeeCoordinateMessage) -> GeeImageResult:
"""Process NDVI imagery for the given coordinates using Earth Engine."""
logger.info(f"Processing NDVI imagery for site {message.siteId}")
try:
self._initialize_ee()
# Create geographic point and buffer
point = ee.Geometry.Point([message.coordinates.longitude, message.coordinates.latitude])
buffer = point.buffer(message.bufferDistance).bounds()
logger.info(f"Buffer created with distance {message.bufferDistance} meters around point {point.getInfo()}")
# Set date range (e.g., last 1 year)
end_date = ee.Date(datetime.utcnow())
start_date = end_date.advance(-message.timeRangeYears, 'year')
# Filter image collection
collection = ee.ImageCollection(message.collection) \
.filterBounds(buffer) \
.filterDate(start_date, end_date) \
.sort('CLOUD_COVER') \
.first()
if collection is None:
logger.warning(f"No {message.collection} imagery found for the location")
return GeeImageResult(
imageType="NDVI",
imageUrl=None,
collection=message.collection.split('/')[-1],
processedDate=datetime.utcnow(),
description="Normalized Difference Vegetation Index - No imagery available"
)
# Determine band names (Sentinel-2 vs Landsat)
if 'S2' in collection.get('system:id').getInfo():
nir_band = 'B8'
red_band = 'B4'
else:
nir_band = 'B5'
red_band = 'B4'
logger.info(f"Using NIR band {nir_band} and RED band {red_band} for NDVI calculation")
# Calculate NDVI: (NIR - RED) / (NIR + RED)
ndvi = collection.normalizedDifference([nir_band, red_band]).rename('NDVI')
logger.info(f"NDVI calculated using bands {nir_band} and {red_band}")
image_url = None
try:
# Try to get thumbnail directly from Earth Engine
thumbnail_url = ndvi.getThumbURL({
'min': -0.2,
'max': 0.8,
'palette': [
'FFFFFF', 'CE7E45', 'DF923D', 'F1B555', 'FCD163',
'99B718', '74A901', '66A000', '529400', '3E8601',
'207401', '056201', '004C00', '023B01', '012E01',
'011D01', '011301'
],
'dimensions': 1024,
'region': buffer,
'format': 'png'
})
logger.info(f"NDVI Thumbnail URL generated: {thumbnail_url}")
# Download thumbnail
async with httpx.AsyncClient() as client:
response = await client.get(thumbnail_url)
response.raise_for_status()
image_data = response.content
img_path = tempfile.mktemp(suffix='.png')
with open(img_path, 'wb') as f:
f.write(image_data)
logger.info(f"Successfully downloaded NDVI thumbnail from Earth Engine")
# Upload to Azure Blob Storage
image_url = await self._upload_or_get_url(img_path, message.projectId, f"ndvi_{message.siteId}", "png")
try:
os.remove(img_path)
except:
pass
except Exception as thumb_error:
logger.warning(f"NDVI Thumbnail failed: {str(thumb_error)}. Falling back to image export")
# Fallback: export as GeoTIFF, then convert to PNG
with tempfile.NamedTemporaryFile(suffix='.tif', delete=False) as tmp:
temp_path = tmp.name
logger.info(f"Temporary file created at {temp_path} for NDVI export")
geemap.ee_export_image(
ndvi,
filename=temp_path,
scale=10,
region=buffer
)
logger.info(f"NDVI image exported to {temp_path} successfully")
img_path = temp_path.replace('.tif', '.png')
self.tif_to_image(temp_path, img_path, apply_colormap=True)
image_url = await self._upload_or_get_url(img_path, message.projectId, f"ndvi_{message.siteId}", "png")
try:
os.remove(temp_path)
os.remove(img_path)
except:
pass
return GeeImageResult(
imageType="NDVI",
imageUrl=image_url,
collection=message.collection,
processedDate=datetime.utcnow(),
description="Normalized Difference Vegetation Index"
)
except Exception as e:
logger.error(f"Error processing NDVI imagery: {str(e)}")
return GeeImageResult(
imageType="NDVI",
imageUrl=None,
collection=message.collection.split('/')[-1],
processedDate=datetime.utcnow(),
description="Normalized Difference Vegetation Index"
)
|
Key archaeological insight: Ancient stone walls buried 30cm underground prevent crop roots from reaching deep soil moisture. During dry summers, these crops show NDVI values of 0.3-0.4 (yellow-brown), while surrounding crops show 0.7-0.8 (dark green). The pattern reveals the wall layout!
The color palette goes: white (bare soil, -0.2) → brown → yellow → light green → dark green (dense vegetation, 0.8).
True color imagery uses standard Red/Green/Blue bands to create a natural-looking image — what you’d see with your eyes from a plane:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
|
async def process_true_color(self, message: GeeCoordinateMessage) -> GeeImageResult:
"""Process true color imagery for the given coordinates using Earth Engine."""
logger.info(f"Processing true color imagery for site {message.siteId}")
try:
self._initialize_ee()
point = ee.Geometry.Point([message.coordinates.longitude, message.coordinates.latitude])
buffer = point.buffer(message.bufferDistance).bounds()
end_date = ee.Date(datetime.utcnow())
start_date = end_date.advance(-message.timeRangeYears, 'year')
# Use Sentinel-2 for true color (10m resolution)
collection = ee.ImageCollection('COPERNICUS/S2_SR_HARMONIZED') \
.filterBounds(buffer) \
.filterDate(start_date, end_date) \
.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 20)) \
.sort('CLOUDY_PIXEL_PERCENTAGE') \
.first()
if collection is None:
logger.warning("No Sentinel-2 imagery found for the location")
return GeeImageResult(
imageType="TrueColor",
imageUrl=None,
collection="Sentinel-2",
processedDate=datetime.utcnow(),
description="True color composite (RGB) - No imagery available"
)
# Sentinel-2 bands: B4=Red, B3=Green, B2=Blue
bands = ['B4', 'B3', 'B2']
image_url = None
try:
thumbnail_url = collection.select(bands).getThumbURL({
'min': 0,
'max': 3000,
'dimensions': 1024,
'region': buffer,
'format': 'png'
})
logger.info(f"Thumbnail URL generated: {thumbnail_url}")
async with httpx.AsyncClient() as client:
response = await client.get(thumbnail_url)
response.raise_for_status()
image_data = response.content
img_path = tempfile.mktemp(suffix='.png')
with open(img_path, 'wb') as f:
f.write(image_data)
logger.info(f"Successfully downloaded thumbnail from Earth Engine")
image_url = await self._upload_or_get_url(img_path, message.projectId, f"truecolor_{message.siteId}", "png")
except Exception as thumb_error:
logger.warning(f"Thumbnail failed: {str(thumb_error)}. Falling back to image export")
with tempfile.NamedTemporaryFile(suffix='.tif', delete=False) as tmp:
temp_path = tmp.name
rgb_image = collection.select(bands)
geemap.ee_export_image(
rgb_image,
filename=temp_path,
scale=10,
region=buffer
)
img_path = temp_path.replace('.tif', '.png')
self.tif_to_image(temp_path, img_path)
image_url = await self._upload_or_get_url(img_path, message.projectId, f"truecolor_{message.siteId}", "png")
try:
os.remove(temp_path)
os.remove(img_path)
except:
pass
return GeeImageResult(
imageType="TrueColor",
imageUrl=image_url,
collection="Sentinel-2",
processedDate=datetime.utcnow(),
description="True color composite (RGB)"
)
except Exception as e:
logger.error(f"Error processing true color imagery: {str(e)}")
return GeeImageResult(
imageType="TrueColor",
imageUrl=None,
collection="Sentinel-2",
processedDate=datetime.utcnow(),
description="True color composite (RGB)"
)
|
Archaeological use: Helps archaeologists orient the site, identify modern features (roads, buildings), and validate LiDAR anomalies against visible landscape features.
False color uses NIR/Red/Green instead of Red/Green/Blue. This makes healthy vegetation appear bright red (high NIR reflection), while bare soil appears cyan/blue:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
|
async def process_false_color(self, message: GeeCoordinateMessage) -> GeeImageResult:
"""Process false color imagery for the given coordinates using Earth Engine."""
logger.info(f"Processing false color imagery for site {message.siteId}")
try:
self._initialize_ee()
point = ee.Geometry.Point([message.coordinates.longitude, message.coordinates.latitude])
buffer = point.buffer(message.bufferDistance).bounds()
end_date = ee.Date(datetime.utcnow())
start_date = end_date.advance(-message.timeRangeYears, 'year')
collection = ee.ImageCollection('COPERNICUS/S2_SR_HARMONIZED') \
.filterBounds(buffer) \
.filterDate(start_date, end_date) \
.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 20)) \
.sort('CLOUDY_PIXEL_PERCENTAGE') \
.first()
if collection is None:
logger.warning("No Sentinel-2 imagery found for the location")
return GeeImageResult(
imageType="FalseColor",
imageUrl=None,
collection="Sentinel-2",
processedDate=datetime.utcnow(),
description="False color infrared composite (NIR/RED/GREEN) - No imagery available"
)
# Sentinel-2 false color: B8=NIR, B4=Red, B3=Green
bands = ['B8', 'B4', 'B3']
logger.info(f"Using bands {bands} for false color visualization")
image_url = None
try:
thumbnail_url = collection.select(bands).getThumbURL({
'min': 0,
'max': 3000,
'dimensions': 1024,
'region': buffer,
'format': 'png'
})
async with httpx.AsyncClient() as client:
response = await client.get(thumbnail_url)
response.raise_for_status()
image_data = response.content
img_path = tempfile.mktemp(suffix='.png')
with open(img_path, 'wb') as f:
f.write(image_data)
image_url = await self._upload_or_get_url(img_path, message.projectId, f"falsecolor_{message.siteId}", "png")
except Exception as thumb_error:
logger.warning(f"Thumbnail failed. Falling back to image export")
with tempfile.NamedTemporaryFile(suffix='.tif', delete=False) as tmp:
temp_path = tmp.name
fc_image = collection.select(bands)
geemap.ee_export_image(
fc_image,
filename=temp_path,
scale=10,
region=buffer
)
img_path = temp_path.replace('.tif', '.png')
self.tif_to_image(temp_path, img_path)
image_url = await self._upload_or_get_url(img_path, message.projectId, f"falsecolor_{message.siteId}", "png")
return GeeImageResult(
imageType="FalseColor",
imageUrl=image_url,
collection="Sentinel-2",
processedDate=datetime.utcnow(),
description="False color infrared composite (NIR/RED/GREEN)"
)
except Exception as e:
logger.error(f"Error processing false color imagery: {str(e)}")
return GeeImageResult(
imageType="FalseColor",
imageUrl=None,
collection="Sentinel-2",
processedDate=datetime.utcnow(),
description="False color infrared composite (NIR/RED/GREEN)"
)
|
Archaeological use: Makes it instantly obvious where vegetation is stunted/stressed due to subsurface features. Ancient ditches (more moisture) appear brighter red. Buried walls (less moisture) appear as dark gaps in red fields.
10. Handling Earth Engine Export Quirks
Earth Engine can return images as thumbnails (fast, 1-2 seconds) or full exports (slow, 20-60 seconds). We try thumbnails first, then fall back to GeoTIFF export if needed.
When we get a GeoTIFF, we convert it to PNG for web display:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
|
def tif_to_image(self, tif_path, out_image_path, apply_colormap=False):
"""Convert a TIF file to a PNG image with normalization and optional colormap."""
try:
from matplotlib import cm
with rasterio.open(tif_path) as src:
band_count = src.count
if band_count == 1:
# Single-band: normalize to 0-255
arr = src.read(1)
if np.isnan(arr).all():
arr = np.zeros_like(arr, dtype=np.uint8)
else:
# Stretch to 2nd and 98th percentile
finite_vals = arr[np.isfinite(arr)]
p2, p98 = np.percentile(finite_vals, (2, 98))
arr = np.clip(arr, p2, p98)
arr = ((arr - p2) / (p98 - p2) * 255).astype(np.uint8)
if apply_colormap:
colormap = cm.get_cmap("viridis")
colored_arr = (colormap(arr / 255.0)[..., :3] * 255).astype(np.uint8)
img = Image.fromarray(colored_arr)
else:
img = Image.fromarray(arr).convert("RGB")
elif band_count >= 3:
# Multi-band RGB: normalize each band
arr = src.read([1, 2, 3])
arr = np.transpose(arr, (1, 2, 0))
for i in range(min(3, arr.shape[2])):
band = arr[..., i]
finite_vals = band[np.isfinite(band)]
if len(finite_vals) > 0:
p2, p98 = np.percentile(finite_vals, (2, 98))
band = np.clip(band, p2, p98)
arr[..., i] = ((band - p2) / (p98 - p2) * 255)
arr = np.clip(arr, 0, 255).astype(np.uint8)
img = Image.fromarray(arr)
img.save(out_image_path)
logger.info(f"Saved converted image to {out_image_path}")
return out_image_path
except Exception as e:
logger.error(f"Error converting TIF to PNG: {e}")
# Create gray placeholder
img = Image.new('RGB', (512, 512), color=(128, 128, 128))
img.save(out_image_path)
return out_image_path
|
This converts single-band images (like NDVI) to colorized PNGs using matplotlib’s “viridis” colormap. For RGB images, it normalizes each band to 0-255 by stretching to the 2nd-98th percentile (removes outliers).
To automate Earth Engine access, we use a service account (like Azure’s managed identity, but for GCP):
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
|
class EarthEngineInitializer:
"""Initialize Earth Engine with service account."""
@staticmethod
def initialize_with_local_service_account():
"""Initialize Earth Engine using local service account JSON file."""
try:
service_account_file = "gee-service-account.json"
if not os.path.exists(service_account_file):
logger.error(f"Service account file not found: {service_account_file}")
return False
credentials = ee.ServiceAccountCredentials(
email=None, # Will be read from JSON
key_file=service_account_file
)
ee.Initialize(credentials)
logger.info("Earth Engine initialized with service account")
return True
except Exception as e:
logger.error(f"Failed to initialize Earth Engine: {str(e)}")
return False
|
The service account JSON is copied into the Docker container at build time. You create this in the Google Cloud Console by:
- Enabling the Earth Engine API
- Creating a service account with Earth Engine permissions
- Downloading the JSON key
- Registering it with Earth Engine (one-time setup)
The FastAPI service runs as a Container App. Here’s the configuration:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
|
class AppConfig(BaseSettings):
"""Application configuration settings."""
# API server settings
host: str = "0.0.0.0"
port: int = 8000
debug_mode: bool = False
# GEE settings
gee_service_account_file: str = "gee-service-account.json"
# Storage settings
storage_account_name: Optional[str] = None
storage_account_key: Optional[str] = None
storage_container_name: str = "satellite-images"
# Logging
log_level: str = "INFO"
@classmethod
def from_env(cls):
"""Load configuration from environment variables."""
return cls(
host=os.getenv("HOST", "0.0.0.0"),
port=int(os.getenv("PORT", "8000")),
debug_mode=os.getenv("DEBUG_MODE", "false").lower() in ("true", "1", "yes"),
gee_service_account_file=os.getenv("GEE_SERVICE_ACCOUNT_FILE", "gee-service-account.json"),
storage_account_name=os.getenv("STORAGE_ACCOUNT_NAME"),
storage_account_key=os.getenv("STORAGE_ACCOUNT_KEY"),
log_level=os.getenv("LOG_LEVEL", "INFO")
)
|
Dockerfile for Container App:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
FROM python:3.11-slim
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
WORKDIR /app
RUN apt-get update && apt-get install -y \
gcc \
libgdal-dev \
curl \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --upgrade pip && pip install -r requirements.txt
COPY . .
# Copy GEE service account credentials
COPY gee-service-account.json .
EXPOSE 8000
CMD ["python", "main.py"]
|
Why Container Apps?
- Scales to zero when not processing (cost-effective)
- Automatic HTTPS endpoints
- Easy environment variable injection (Azure Storage keys, etc.)
- Built-in health checks and logging
Key dependencies:
earthengine-api - Google Earth Engine SDKgeemap - Simplifies EE image exportrasterio - GeoTIFF reading/processingfastapi + uvicorn - Web serverPillow - Image format conversion
After satellite imagery processing completes, the orchestrator passes all three image URLs to the multi-agent AI workflow. Here’s what each agent uses:
TerrainSpecialist Agent:
- Analyzes LiDAR hillshade + NDVI together
- “The hillshade shows a 2m elevation change forming a rectangle. NDVI shows vegetation stress in the same pattern. High confidence this is a buried structure.”
EnvironmentSpecialist Agent:
- Studies true color + false color
- “False color shows modern drainage patterns in the northeast. True color confirms this is active farmland. The site is accessible but cultivation may have damaged shallow archaeology.”
ArchaeologyAnalyst Agent:
- Combines all imagery types
- “NDVI stress pattern aligns with LiDAR mound. True color shows no modern structures. False color indicates moisture retention consistent with ancient ditch systems. Recommend excavation.”
The satellite imagery enriches the AI analysis by adding environmental and vegetation context that LiDAR alone can’t provide.
🌍 Satellite imagery adds environmental context that LiDAR elevation data can’t provide alone
🌾 NDVI reveals vegetation stress caused by buried walls, ditches, and foundations
🔴 False color infrared makes subsurface features visually obvious
⚡ Parallel processing (Task.WhenAll) reduces total processing time from 2-3 minutes to ~60 seconds
🐍 Decoupled Python service allows Earth Engine integration without bloating the .NET orchestrator
🤖 Multi-agent AI uses all three imagery types to make archaeological assessments
Ready to integrate Google Earth Engine with Azure workflows?
🔗 GitHub Repository: https://github.com/Cloud-Jas/Archaios
📚 Key Files to Study:
🚀 Quick Start:
1
2
3
4
|
git clone https://github.com/Cloud-Jas/Archaios.git
cd Archaios/src/backend/Archaios.AI.GeeProcessor
pip install -r requirements.txt
python main.py
|
Integrating Google Earth Engine’s satellite imagery catalog with Azure enables powerful remote sensing analysis for archaeology and beyond. If you’re working with geospatial data or building Earth observation workflows, let’s connect on LinkedIn!
#GoogleEarthEngine #Azure #Python #RemoteSensing #NDVI #Archaeology #Geospatial #MicrosoftMVP
