Is there any reason to prefer functional programming for data mining projects? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.

---

Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.

Closed 7 years ago.

开发者_StackOverflow社区 Improve this question

I am researching the possibility of starting a data mining project which will include intensive calculations and transformation on data, and should be relatively easy to scale.

In your experience, is the choice of programming language critical for said project?

For example, if I am already working on a JVM environment, should I prefer Clojure over plain Java? Does the functional environment guarantee easier scalability? Better performance?

Put aside other factors such as familiarity with the language, toolchain, etc. In your experience, is the choice of language a critical one?

---

There are a few good reasons for choosing functional programming for data mining projects.

1. Usually data mining projects involve algorithmics and mathematics (than other types of systems) which can be more easily expressed in functional programming
2. Data mining projects would involve aggregate functions - which are better in functional programming, say Clojure
3. Data mining programs also would be more suitable to parallelism - definitely data parallelism and could even be task parallelism, again a forte of functional programming
4. And functional languages like Clojure can interface with java anyway for I/O, file read and write
5. I think one can learn the tool chain easily; it is not that different and so that shouldn't be a factor.

I was asking the same question myself and came with a big Yes for Clojure - am still thinking through how to include R in the mix.

---

Use the most powerful language you are comfortable with.

In any case, if you want to get scalability you need to have a map-reduce implementation which allow you to parallellize and collect the results.

---

No particular reason. Pick whatever language you feel most comfortable with.

See my answer to a similar question about natural language processing. I think that some of the features people think obscure languages are suited to AI are really counterproductive.

---

Often, functional programming solutions are more scalable.