Which model to use, and where

Measured on the six annotated chips and the three wider pasture scenes in this bench. Object scoring is one-to-one at IoU > 0.5 in an equal-area projection.

Verdict

inv3_b08 for pasture, masked to MapBiomas agricultural classes before polygonization. inv2_b07 for oil palm.

What separates the four models

They differ only in the per-class Tversky weights on the boundary channel: alphas[2]/betas[2] of 0.5/0.5 for r0_sym, then 0.4/0.6, 0.3/0.7 and 0.2/0.8. Note that smp.losses.TverskyLoss reads alpha as the false-positive penalty and beta as false-negative, which is the reverse of the comments in the configs, so the ramp trains boundary recall. On one pasture chip the boundary share of the argmax rises from 16.7% to 25.7% across the ramp and the raw blob count goes 46 to 81. Every scene follows: more polygons, smaller, at every step.

The extra cuts are real

Taking only the polygon edges that fall strictly inside an annotated field, and comparing the image gradient under them to the gradient of the field interior, the internal cuts score 2.39 for r0_sym and 2.25 for inv3_b08 over the six chips. An invented line would score about 1.0. inv3_b08 subdivides annotations that were drawn as single management units, which one-to-one IoU scoring charges as false positives. On pasture it matches 28 fields r0_sym misses and loses 11, a net gain of 17.

Pooled pasture accuracy, three annotated chips

Annotations floored at 0.4 ha to match the 4,000 m² floor the predictions already carry. inv3_b08 leads on recall by nearly a third.

Pooled oil palm accuracy, three annotated chips

Same floor. inv1_b06 leads the F1 column, but that lead is an artifact of the threshold and the pick is inv2_b07. Detection is a four-way tie: of 172 annotated fields, r0_sym matches 51, inv1_b06 54, inv2_b07 52 and inv3_b08 53, with pairwise swaps of 7 and 9. inv1's edge comes from drawing fewer polygons, 112 against 135, which one-to-one matching rewards. Score the outlines instead, with best IoU per field and no cliff, and inv2_b07 leads at 0.358 mean and 0.237 median against inv1_b06's 0.343 and 0.177 — a third better at the typical field. inv1_b06 is also the worst of the four at staying on crop, with 22 of its 112 polygons on no annotation at all, against 21 of 135 for inv2_b07.

Where it fails

Pushing boundary recall makes the model carve texture in land that holds no fields. In palm forest that is severe: 8 of 52 polygons sit on no annotation at all for inv3_b08 against 2 of 42 for r0_sym, with closed canopy shredded into field-shaped pieces. On pasture chips it nearly vanishes, 7 of 245 polygons against 2 of 156. In the wider pasture scenes, polygons whose majority MapBiomas 2024 class is not pasture, cropping or mosaic run 3.4%, 5.9% and 9.3% for inv3_b08 against 1.5%, 4.5% and 3.5% for r0_sym. The 9.3% is Western Bahia, 161 polygons over 416 ha of native Cerrado. Masking to agricultural classes before polygonization removes this, and the MapBiomas layer is already loaded for scene selection.

How the scoring was corrected

Annotations are clipped to the chip frame, because the model only sees the chip and full polygons would charge every edge field as a miss. Two artifacts came with that. Clipped remnants were kept down to 100 m² while predictions are floored at 4,000 m², so 69 of the 456 annotations could not be matched by any model under any setting; applying the same floor to both sides lifts recall for all four and widens inv3_b08's pasture lead from 0.318 to 0.352 F1. A fifth of what remains, 102 polygons, is truncated by the frame, so the truth areas quoted here are chip areas rather than field areas. Dropping truncated fields as well costs so much of a 2.3 km chip that the pooled counts halve, which is a limit of the chip size rather than a result.

One wider area, scored

Annotations are not confined to the chips. 530 of the 2,029 pasture polygons, 43% of the annotated hectares, fall outside every chip footprint. For pasture_01 that puts 29 extra fields inside the wider frame, all within 650 m of the chip edge, so the chip frame grown by 700 m gives 96 scoreable fields. Recall over them runs 0.229 for r0_sym, 0.198 inv1_b06, 0.260 inv2_b07 and 0.312 for inv3_b08, the same order as the chips. Precision there, 0.11 to 0.19, is not usable: the ring is only partly annotated, and the wider imagery is a different date pair from the one the fields were drawn on. Oil palm has no such margin, with only 27 of 1,297 polygons outside a chip.

Two trained fixes, both refused

clDice is a connectivity loss meant to penalize gaps in thin structures, and CustomTverskyClDiceLoss pinned it to class 1, the field interior, rather than class 2, the boundary. Pointing it at the boundary and retraining made things worse, not better. ava_sa_ct_cld2, which is r0_sym plus the corrected clDice at weight 0.2 and nothing else, scores pasture F1 0.296 against r0_sym's 0.340 and palm F1 0.229 against 0.374, with field-level IoU on palm of 0.206 against 0.315 for the worst baseline. ava_sa_ct_cld2_thin, which adds alphas [0.5, 0.3, 0.5] and betas [0.5, 0.7, 0.8] on top, scores pasture F1 0.269 and palm 0.245. Both predict far too few polygons — 56 and 57 across the three palm chips where the baselines predict 101 to 135, with false positives going down and false negatives up. That is merging, not error. The older cldice_r1 run showed the same signature with the term on the other class, so the class was never the problem: this clDice implementation suppresses subdivision whichever channel it skeletonizes. The conclusion to carry forward is that the connectivity loss should stay off until someone works out why, and that the two-channel Tversky push has still not been tested on its own.

What none of them fix

False positives almost never land outside annotated ground. Every model finds the right fields and draws the wrong number of lines on them, so the remaining error is granularity, not detection. Moving further up this ramp will not change that.

Field Boundary Model Bench

Four boundary-loss variants decoded with BoundaryVote and a 4,000 m² floor, over MapBiomas 2024 pasture and oil palm in Brazil.

Scene

Imagery

Model layers — click to solo, shift-click to add

Accuracy on the annotated chip

Scene detail