fix compatibility + fix #95

[jeffreyhanson]

• Update wdpa_read() and wdpa_clean() to be compatible with updates to
  the Protected Planet database. Thanks to Joe Gosling for providing detailed
  information on the database changes.
• Update wdpa_clean() so that it shows clearer progress messages for erasing
  spatial overlaps, throws a warning if attempting to erase overlaps from
  particularly large datasets, trims white space characters from all
  fields containing character values, and can optionally skip steps for
  repairing geometries if specified by the user (per the new
  repair_geometries parameter) (feature request: make geometry repair optional in wdpa_clean() #95). Thanks to Alexandre Courtiol
  (@courtiol) for feature suggestion.

[courtiol]

FYI, after pulling your PR and running R CMD check on my laptop I noticed that the vignette would also need to be updated, turning:

• MARINE into REALM (4 times)
• terrestrial into Terrestrial (twice)
• marine into Marine (twice)
• WDPAID into SITE_ID (twice).

Once this is done, the vignette runs fine, but I did notice that numbers returned are different from those returned in the original vignette (judging from the website).

For example, the histogram displaying the STATUS_YR does not include the early observation; one polygon appears as Not Reported while it was under a IUCN category in the previous map of Malta; the total area is different too.

I did not look into whether this is just because underlying data have been updated or whether something else is going on.

Otherwise the checks run fine!

[jeffreyhanson]

Thanks so much for checking this PR and catching those mistakes @courtiol - I really appreciate it!

[jeffreyhanson]

After some digging, it seems the reason this passed locally was because I had cached an older version of the WDPA.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.80%. Comparing base (a7c7b76) to head (4689bdd).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #96      +/-   ##
==========================================
- Coverage   97.69%   96.80%   -0.90%     
==========================================
  Files          10       10              
  Lines         608      626      +18     
==========================================
+ Hits          594      606      +12     
- Misses         14       20       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[courtiol]

You are very welcome.
I see another small miss, within wdpa_clean and perhaps elsewhere you still use PARENT_ISO3 but the new name is PRNT_ISO3.

[courtiol]

I guess this all block is now useless (since MARINE no longer exists and REALM is coded with levels), unless you are trying to have your code work on different version of WDPA, but I think you are not.

[courtiol]

Another issue is that not fixing geoms exposed this bug: r-spatial/lwgeom#100
I expect it to be fixed soon.

Update now fixed in lwgeom devel (9 min after posting the issue!), but I noticed in the meantime that the empty polygon do not come from WDPA but are generated by wdpa_clean(). This occurs during the 0 buffering step when applied to SITE_ID 389369 which has a wrong polygon coming from WDPA (looks like 2 lines making a right angle, i.e. a corner, see below). I think that Protected Planet should fix this, not you... I reported it to them.

[jeffreyhanson]

You are very welcome. I see another small miss, within wdpa_clean and perhaps elsewhere you still use PARENT_ISO3 but the new name is PRNT_ISO3.

I guess this all block is now useless (since MARINE no longer exists and REALM is coded with levels), unless you are trying to have your code work on different version of WDPA, but I think you are not.

Thanks for taking another look at this! Yeah, my goal was to update wdpar so that it would be compatible with the new version of the WDPA, as well as maintain backwards compatibility (where possible) with older versions of the WDPA. As such, I've updated the code for formatting the PARENT_ISO3 and MARINE columns so that the formatting should only be applied if these columns are present.

https://github.com/prioritizr/wdpar/blob/fix-compat/R/wdpa_clean.R#L359-L361

https://github.com/prioritizr/wdpar/blob/fix-compat/R/wdpa_clean.R#L502-L507

How does that sound?

[jeffreyhanson]

Another issue is that not fixing geoms exposed this bug: r-spatial/lwgeom#100 I expect it to be fixed soon.

Update now fixed in lwgeom devel (9 min after posting the issue!), but I noticed in the meantime that the empty polygon do not come from WDPA but are generated by wdpa_clean(). This occurs during the 0 buffering step when applied to SITE_ID 389369 which has a wrong polygon coming from WDPA (looks like 2 lines making a right angle, i.e. a corner, see below). I think that Protected Planet should fix this, not you... I reported it to them.

Thanks for catching this bug as well! Yeah, although I could try adding some extra functionality to identify records like this that are associated with geometries that are most likely incorrect, I agree that it would be better for Protected Planet to resolve this if possible. The wdpa_clean() function does remove slivers though (i.e., currently defined as polygons smaller than 0.1 m^2), and perhaps future versions could also consider using area-to-perimeter ratios as well.

[courtiol]

Thanks for trying to make it backward compatible. I just wish WDPA had not change column names, but that is not on you. The decision to have wdpar returning the WDPA names of the data it reads does make sense. I did not try the new fork on old data, but I guess you did. I think that polygon issues will continue to be progressively better handled by sf, lwgeom, etc... which packages like yours use. So, it seems that it could be ready to merge 🚀

[jeffreyhanson]

Ok great - thanks again for all your help with this!