Just as CAST::CreateSpacetimeFolds() does.

I am not sure if this approach can work with all currently implemented spatial sampling methods. Even if not, we should support exactly this way of creating resamplings since some people already asked me exactly for this. @HannaMeyer Is there a dedicated name for your method? If not, do you want to make a proposal? :) You can have a look at the current names of the other methods in the README.

It seems that @jannes-m has added temporal extension support for spcv-coords already. Let's have a look how this works in detail.

If the target has a binary outcome, a presence-background approach (see blockCV::buffering) would be possible. Target needs to be transformed to 0/1 before sampling.

autoplot throws the following warning when some of the resampling methods are applied to the ecuador task (see application of methods "spcv_disc" and "spcv_block" in the manuscript):

CRS not set, transforming to WGS84 (EPSG: 4326).

This doesn't seem to make sense, as a transformation to WGS84 is not possible when the source CRS is unknown. As far as I can see the ecuador dataset and task contains only UTM coordinates, not even lat/lon, therefore it is also not possibly to just assume that WGS84 is present or to guess which UTM zone is applicable. Just a minor issue, but potentially confusing...

Dear mlr3 team,

first of all, thanks for your efforts in developing this extension package, it is very much appreciated.

I am trying to apply spatial CV using "spcv_coords" to an AutoTuner in order to retrieve nested resampling following the process described in the mlr3 book

        RT.at_sp <- AutoTuner$new(
          learner = reg.tree,
          resampling = spatial_CV, 
          measure = opt.mse,
          search_space = param_set_RT,
          terminator = trm.evals,
          tuner = tnr.GridSearch)

However, I end up with the error message:

        "Error: Resampling 'spcv_coords' may not be instantiated". 

The same error message remains, even if I try to instantiate the task manually beforehand using the command

        spatial_CV$instantiate(sp_task)

as described in 2.5.2.

As I am not an expert, do I make something wrong, or is spatial CV not yet implemented for use with AutoTuner?

Thank you very much! BR, Jürgen

When using spatial object with unprojected CRS (i.e. lat/lon), does mlr3spatiotempcv use great circle distance (on the ellipsoid) or Euclidean distances based on lat/lon values? Is this handled consistently across resampling tools, e.g. buffering and clustering? This should be clarified in the paper, but perhaps it should be turned into a feature request...

#6

plot.ResamplingSpCVBlock, plot.ResamplingSpCVEnv, plot.ResamplingSpCVKmeans are the same. We could create a super class to just have one plotting function. plot.ResamplingSpCVBuffer is different because it is a leave-one-out cross-validation, which cannot be visualized in the same way.

• [x] Single fold plot

• [x] Single train-test plot

• [x] Multi train-test plot

• [x] Unify redundant code

• [x] Update documentation

• [x] add tests

Examples are in inst/mlr3spatiotemporal_test.R at the end of the file.

Dear mlr3spatiotempcv team,

First, many thanks for your hard work on this excellent resource.

I am having an issues producing a checkerboard sampling pattern using spcv_block. Instead of getting a checkerboard spatial partitioning, I always get something that looks more like a random sampling pattern. I have been successful creating a checkerboard pattern using the blockCV functions directly.

Here is a reproducible example that fails to produce a checkerboard sampling pattern:

library(blockCV)
library(mlr3)
library(mlr3spatiotempcv)

x <- runif(5000, -80.5, -75)
y <- runif(5000, 39.7, 42)

data <- data.frame(spp="test", 
                   label=factor(round(runif(length(x), 0, 1))),
                   x=x,
                   y=y)

testTask <- TaskClassifST$new(id = "test", 
                              backend = data, 
                              target = "label",
                              positive="1",
                              extra_args = list(coordinate_names=c("x", "y"),
                                                crs="EPSG: 4326"))

blockSamp <- rsmp("spcv_block",
                  folds=2,
                  range=50000,
                  selection="checkerboard")
blockSamp$instantiate(testTask)
autoplot(blockSamp, testTask)

• Be reasonable with dependencies. E.g., we do not need stringr for str_detect, just use grepl() instead.

• Some examples are in DONTRUN. Can we put them in if (requireNamespace(...)) blocks instead?

• ResamplingSpCVBuffer looks similar to LOO. If this is the case, the instance should be stored more efficiently.

• autoplot tests are not working for me or are waaay to slow for unit tests (I'm stuck in there with 100% CPU)

• blockCV::spatialBlock() seems to call print(). It is convention (maybe even a CRAN policy?) to use message() which you can be suppressed. This should be reported upstream.

• blockCV is terrible slow (and has a stupid long dependency chain). Is there an alternative?

• Suggested packages should be explicitly attached with require_namespaces.

library(mlr3spatiotempcv)
library(mlr3)
rsp <- rsmp("repeated_spcv_coords", folds = 3, repeats = 5)
rsp$instantiate(tsk("ecuador"))

# should return 3
rsp$folds(6)
#> [1] 1

^{Created on 2021-04-17 by the reprex package (v2.0.0)}

This is because in https://github.com/mlr-org/mlr3spatiotempcv/blob/b9ded4ac098655dc00c300b48426bd6d4cd0a97a/R/ResamplingRepeatedSpCVCoords.R#L54 %% is used whereas it should be %/%.

But there is more to it - I think the method should look like

    folds = function(iters) {
      iters = assert_integerish(iters, any.missing = FALSE, coerce = TRUE)
      n_folds = ((self$iters - 1L) %/% as.integer(self$param_set$values$repeats)) + 1L

      if (all(iters <= n_folds)) {
        return(iters)
      } else {
        # modify all entries which are > n_folds
        iters[which(iters > n_folds)] = iters[which(iters > n_folds)] - n_folds
        return(iters)
      }
    }

• [x] add some use cases for each method
• [x] list more implementations for each method (eventually also in other languages?)

Also consider to move methods operating in the feature space into a distinct group, e.g.:

• spatial
• spatiotemporal
• feature space

• arguments crs, coordinate_names and coords_as_features are now passed directly in the constructor instead of list extra_args
• added argument label
• improved as_task_* converters
• Column role coordinates was renamed to coordinate to cope with the singular naming of column roles
• Task printer only returns the first 10 coordinate rows
• Consolidated autoplot() code internally
• Improved CLUTO test setup

fixes #116

This package depends on (depends, imports or suggests) raster and one or more of the retiring packages rgdal, rgeos or maptools (https://r-spatial.org/r/2022/04/12/evolution.html, https://r-spatial.org/r/2022/12/14/evolution2.html). Since raster 3.6.3, all use of external FOSS library functionality has been transferred to terra, making the retiring packages very likely redundant. It would help greatly if you could remove dependencies on the retiring packages as soon as possible.

We could support this via the ellipsis. Otherwise setting the respective column roles could be somewhat easily forgotten. On the other hand behaviour would differ than compared to as_task_*() from mlr3 as such custom conversions would not be supported there.

@be-marc what do you think?

Example:

data("cookfarm_sample", package = "mlr3spatiotempcv")

# data.frame
as_task_regr_st(cookfarm_sample, target = "PHIHOX",
  coords_as_features = FALSE,
  crs = 26911,
  coordinate_names = c("x", "y"),
  column_role_space = "foo",
  column_role_time = "time"
)

```

With the addition of spatial DataBackends (DataBackendVector and DataBackendRaster) from {mlr3spatial} multiple combinations of Tasks and Backends are possible:

• Task + spatial backends
• Task*ST + non-spatial backend
• Task*ST + spatial backend

Moving forward and to simplify both usage and development, we should pick one combination as the "recommended" one and potentially issue warnings for others.

cc @be-marc

Unfortunately the GH repo leads to a 404. Contacted the author, he wants to fix it.

https://www.sciencedirect.com/science/article/abs/pii/S0920410521015023

I currently have a task with a column that is a date. As the task is to basically predict values in the future, a cross-validation strategy that can take this into account would be required. Similar to see RollingWindowCV. As this is a very common use-case, we should perhaps think about implementing this.

• This is implemented in mlr3forecasting, but for forecasting tasks instead of regular Classif|Regr Tasks.
• Where should such a method live? mlr3spatiotempcv ?
• How would we go about implementing this.