开发者

mysterious error from foreach

开发者 https://www.devze.com 2023-02-07 13:42 出处:网络
On my first foray into foreach I am getting this error: Error in function (handle): no function to return from, jumping to top level

On my first foray into foreach I am getting this error:

   Error in function (handle)  :
  no function to return from, jumping to top level

I get one of these for each worker.

I am using the doMC/multicore back end.

At first I thought this was caused by the t开发者_如何转开发ext progress bars that the function inside my foreach call was trying to send to the console as a side effect, but I still get the error after turning that off. Despite the errors the evaluation continues and all hamsters appear to be running. The function inside happens to be reclass() calc() from library(raster). Can anyone surmise what this might mean? I thought I would ask while I wait to see whether the output is valid.


Update

here's the heart of the code:

    foreach( cover= names( classes), .packages= "raster") %dopar% {
      class <- classes[[ cover]]
      calc( mlct$pri,
           function( pri) {
             ifelse( is.na( pri), NA,
                    ifelse( pri ==class, 1, 0))
           },
           filename= paste(
             mlctName,
             paste( cover, ".tif", sep=""),
             sep="_"),
           overwrite= TRUE, ...)
    }

In case of unfamiliarity with library( raster) it provides a clever mechanism for interacting with geospatial raster data sets without having to snarf them into memory. The functions read and write data block by block as they go. In this case the workers are all reading from the same source data but writing to separate output files. I wonder if this file I/O has anything to do with the (seemingly benign) errors reported above.


Now it has spit out all of the new files that I expected but this appears on the console:

GDAL Error 1: TIFFFetchDirectory:Sanity check on directory count failed, this is probably not a valid IFD offset

and the interpreter prompt is not reappearing despite the fact that the cores are quiet. Another clue that file I/O has gone awry somehow, I would guess. Any thoughts?


I think I have the answer. I got some help from Even Rouault on the GDAL-dev list. See this thread in the GDAL-dev archives (also available through Nabble). If he ever comes along he should get credit for this answer, not I.

The crux of the matter is that I have to pass filenames of raster data sets to foreach workers and re-instantiate the raster() objects within the foreach closure. Each worker needs to have its own rgdal handle for reading the input data.

The new code looks like this:

  fracDoparFun <- function( priFilename, ...) {
    foreach( cover= names( classes), .packages= "raster") %dopar% {
      class <- classes[[ cover]]
      frac <-
        calc( raster( priFilename),
             function( pri) {
               ifelse( is.na( pri), NA,
                      ifelse( pri ==class, 1, 0))
             },
             filename= paste(
               mlctName,
               paste( cover, ".tif", sep=""),
               sep="_"),
             overwrite= TRUE, ...)
      return( filename( frac))
    }
  }
mlct$fracs <-
  brick( stack( fracDoparFun( filename( mlct$pri), ...)),
        filename= fracsBrickFile,
        overwrite= TRUE,
        ...)

Now fracDoparFun() returns a list of filenames that are the outputs generated by the workers that are then collected into a single multi-band result by the brick() function.

Thanks for taking a look everyone. I hope I'm not closing this prematurely, but I have tested it and I wanted to capture what I had learned.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号