Skip to contents

Development instructions

Source: AGENTS.md

If you are a human, take these instructions as guidance. If you are an agent, follow these instructions. Ask for clarifications where specifications are unclear or edge cases crop up.

This is an R package that is used for land use / land cover change analyses and simulations. It builds on the approaches of lulcc, clumpy, and Dinamica EGO.

Project layout

  • R/ducklake_db.R, R/ducklake_db_utils.R: generic DuckLake-backed DB layer
  • R/evoland_db.R, R/evoland_db_util.R, R/evoland_db_views.R: domain DB and views (inherits ducklake_db); register new methods using create_method_binding in evoland_db constructor
  • R/*_t.R: table type definitions (as_*_t constructors)
  • R/util*.R: internal helpers
  • R/*.R (remaining): domain logic (allocation, filtering, modelling)
  • src/: Rcpp / C++ code
  • inst/tinytest/: tests
  • inst/dinamica_models/: Dinamica EGO models
  • inst/*.sql: SQL queries, interpolated with glue::glue_sql() by $get_query()
  • data-raw/: raw data and test fixture generation scripts
  • man/: documentation generated by roxygen2
  • vignettes/*.qmd: vignettes and how-tos
  • devdocs/: developer documentation; design rationale in devdocs/design/, style guide in devdocs/style/

Environment setup

Use rv. Run rv sync to install all dependencies (see rproject.toml). Recompile and attach the package using pkgload::load_all().

Conventions and Style

Developer documentation is in devdocs/: design rationale in devdocs/design/, the style guide in devdocs/style/. Before writing or editing a file, read the part of the style guide that applies:

Editing Read
R code in R/, tests in inst/tinytest/ devdocs/style/r.md
C++ in src/ devdocs/style/cpp.md
SQL in inst/*.sql or inline in R devdocs/style/sql.md
Vignettes, roxygen text, Markdown, commits devdocs/style/prose.md

The rules that apply everywhere:

  • Avoid new dependencies; prefer base R, and avoid the tidyverse (unstable APIs) and niche, seldom-maintained packages.
  • Comment why, not what. In R, prefer an unbroken pipe with a short comment over naming intermediate results that are used only once.
  • Let errors propagate naturally; only add early guards when the default error message would confuse the user.
  • Format what you change: air format . for R, git clang-format for C++, sql-formatter for SQL.
  • Don’t reformat or rewrite code unrelated to your change.
  • Use descriptive git branch names; don’t use container or session names.

Testing

Use tinytest, see inst/tinytest. Non-exported functions need to be tested as evoland:::private_function, because a package can be tested after installation. Be parsimonious when constructing tests. Keep test_<table>_t.R files free of database logic: they test the as_*_t constructors and validation only. Tests that need a database belong in test_db_*.R; extend an instance those files already open rather than creating a new one. Tests that need a longer series of manipulations of the database (fitting models, allocating periods, evaluating runs) belong in test_integ_*.R.

  • Test the full package using R -e "tinytest::build_test_install()".
  • Test individual files using R -e "pkgload::load_all(); tinytest::run_test_file('inst/tinytest/somefile.R')"

Documentation

Use markdown roxygen for function and class documentation. Use R -e "roxygen2::roxygenize()" to synchronize roxygen comments and .Rd files. Writte tutorials and how-to as quarto files in vignettes/*.qmd.

Rcpp components

C++ code is in the src/ folder. This code interfaces with R using Rcpp. Build binaries using pkgbuild::build(). Clean build objects using pkgbuild::clean_dll(). Prefer Rcpp types. Only write Rcpp-free headers when the code should also compile as a standalone program.

Database

  • Storage is a DuckLake catalog attached to an in-memory DuckDB instance. By default a database is one self-contained folder: SQLite catalog at <path>/catalog.sqlite, data files under <path>/data/. catalog and data_path move either half elsewhere (PostgreSQL, S3); the extensions needed to open a database are derived from them.
  • R/ducklake_db.R specifies an R6 class that provides database operations to write and retrieve data.table objects. Writes are atomic and readers get snapshot isolation, so several processes may write to one database concurrently; a private retry wrapper absorbs catalog lock contention, which DuckLake’s own ducklake_max_retry_count does not cover. The errors it treats as transient are catalog-backend specific – see TRANSIENT_CATALOG_ERRORS.
    • ducklake_db_t is a subclass of the data.table S3 class, see R/ducklake_db_utils.R
    • ducklake_db_t objects can hold attributes used to define
      • key columns, i.e. uniqueness columns
      • hive partitioning columns, which are a pruning hint only
      • map columns, i.e. R list columns of named lists translated to DuckDB MAP columns
    • Which columns serve which purpose is read from those attributes on the data being committed, and otherwise from the metadata the target table already carries – ducklake_db makes no assumption about how domain constructors are named.
    • DuckLake supports no constraints, keys or indexes, and MERGE silently inserts duplicates from a source with duplicate keys, so $commit() checks uniqueness explicitly before merging. It has no ENUM type either; factors are stored as strings and cast back by the as_*_t() constructors.
  • Domain specific database elements are in R/evoland_db.R; evoland_db inherits from ducklake_db.
    • The schema for this database is (for now) distributed across the class definitions: all R/*_t.R files contain as_*_t class constructors using as_ducklake_db_t.
    • Because the catalog may be read and written from external tools, it should be considered part of the API. Schema changes should be avoided as much as possible.
    • After changing a table’s columns or keys, regenerate the ER diagram with Rscript devdocs/er_diagram.R; CI fails if it is out of date.
    • Ad-hoc views are suffixed _v and generally exposed as active bindings, or as methods if they are parameterized.
    • Every new method must be added in R/evoland_db.R and not with $set. This is because the roxygen documentation routine for R6 objects relies on all documentation being available in a single file.