I have one Java Client which Reads 4 to 5 GBs of data and uses DLL file in order to process it and give O/P back to the JAVA client. Can anyone tell me if this can be implemented using Apache Spark or not ? And if yes can you back you Answer by some examples or documentation Links ?


I have knowledge about JNI and how to do this in windows environement. What I'm actually looking for is How do you implement such thing in Apache Spark.

As all methods and processing is done via DLL files and JAVA Client is just having represntational logic so any documentation is helpful.

I'm currently using windows development env. but gonna use Ubuntu 14.04 for Apache Spark implemetation.

EDIT after Hold

This question is really intending to ask where to find such documentation or help with which we can migrate such application smoothly.

  • $\begingroup$ @Sean Owen Kindly can you explain the reason why you put this question on hold as it is sort of subjective question and I can quote such question from SO only where it is still not answered. $\endgroup$
    – spt025
    Aug 19 '15 at 19:14
  • $\begingroup$ I migrated it to StackOverflow, since it's about developing a software application. $\endgroup$
    – Sean Owen
    Aug 20 '15 at 7:09

You will need to use the Java Native Interface (JNI) for this. Refer to a Stack Overflow question for how to call DLL methods via Java over JNI.

  • $\begingroup$ I'm using JNI in my projects and lets say I'm (not much but) comfortable in that By now. But if we are doing real processing of data via Some DLL methods only then is it wise to use Apache Spark and how do we use it optimally ?? $\endgroup$
    – spt025
    Aug 19 '15 at 11:52
  • $\begingroup$ I know it is application specific, still some guideline or some documentation I'm looking for. Apart from SO questions there is no definitve Guideline is availabe as far as I saw. $\endgroup$
    – spt025
    Aug 19 '15 at 11:53
  • $\begingroup$ @spt025 Could you edit your question to add some specific concerns? I'm not sure if you asking about concurrency, performance, or technical implementation. $\endgroup$ Aug 19 '15 at 11:54
  • $\begingroup$ Yea sure , Will do that .. $\endgroup$
    – spt025
    Aug 19 '15 at 11:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.