开发者

How to perform decision tree lookup using mapreduce? I am looking for an optimized version

开发者 https://www.devze.com 2023-02-27 05:55 出处:网络
I have decision tree with millions of nodes, serialized on HDFS. Can any one please help me giving some pointer how to do better serialization so tha开发者_StackOverflowt I can perform search more eff

I have decision tree with millions of nodes, serialized on HDFS. Can any one please help me giving some pointer how to do better serialization so tha开发者_StackOverflowt I can perform search more efficiently on Hadoop using map reduce.

Thanks.


Well in order to traverse your tree, you need the model to be loaded into memory. Once it is loaded it is pretty easy and fast to perform a traverse of an instance. You cant avoid storing your model into hdfs, so in order to perform a better traverse, you need to do something better in your main memory. But as i said, a tree traverse is always super fast. Perhaps providing some more information about your problem would be nice. Your problem is having millions of new examples and predicting their label?

0

精彩评论

暂无评论...
验证码 换一张
取 消