开发者

hadoop mapper static initialisation

开发者 https://www.devze.com 2023-01-02 10:02 出处:网络
I have a code fragment in which I am using a static code block to initialize a variable. public static class JoinMap extends

I have a code fragment in which I am using a static code block to initialize a variable.

public static class JoinMap extends 
            Mapper<IntWritable, MbrWritable, LongWritable, IntWritable> {
        .......
        public static RTree rt = null;
        static {
            String rtreeFileName = "R.rtree";
            rt = new RTree(rtreeFileName);
        }
        public void map(IntWritable key, MbrWritable mbr,Context context)
                throws IOException, InterruptedException {
                            .........               
                List elements = rt.overlaps(mbr.getRect());
                .......

        }

    }

My problem is that the variable rt in the above code fragment is not getting initialised. Can anybody suggest a fix or an alternate way to initialise the variable. I don't want to initialise it inside my map function 开发者_如何学JAVAsince that slows down the entire process.


That really isn't possible, unless Java itself is broken. The static initializers always fire at class load time.

Maybe whatever you're observing has an alternate explanation, like, something is setting rt back to null. Or, are you observing rt in other statically-initialized expression above? it would see null until rt's initialization finishes.

But, nevertheless, I'd say it's tidier to override the setup() method (configure() in the old API) and do initialization there. It'll happen once.


public static class JoinMap extends 
        Mapper<IntWritable, MbrWritable, LongWritable, IntWritable> {
    .......
    public RTree rt = null; //note that i removed static modifier

    @Override
    public void setup(Context context) throws IOException {
        //this will be executed once on each mapper before first map(..) call
        String rtreeFileName = "R.rtree";
        rt = new RTree(rtreeFileName);
    }

    public void map(IntWritable key, MbrWritable mbr,Context context)
            throws IOException, InterruptedException {
                        .........
    }

}


Is there some reason you don't want to initialize rt like this?:

public static RTree rt = new RTree("R.rtree");


I'm no Java expert but it look like you have two "static" assignments for the variable rt: You have:

public static RTree rt = null;

AND

rt = new RTree(rtreeFileName);

In which order are these assigments done?

Try this instead and see if it helps

    public static final RTree rt = new RTree("R.rtree");

As far as my Java knowledge goes this assures you'll have only single assignment.


what if we don't declare a static block and just write.

public static RTree rt =  new RTree(rtreeFileName);

because the variable is static it will be initialized once when class is loaded in memory?


you can override the setup() method of mapper class to initialize anything in regards with map function. This setup() method is called only once, in the beginning of every mapper,and so, it will not slow down the mapper process

0

精彩评论

暂无评论...
验证码 换一张
取 消