Questions tagged [tools]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
52
votes
8answers
13k views

Why do internet companies prefer Java/Python for data scientist job?

I see a many times in job description for data scientist asking for Python/Java experience and disregard R. Below is a personal email I received from chief data scientist of a company I applied for ...
50
votes
9answers
8k views

Tools and protocol for reproducible data science using Python

I am working on a data science project using Python. The project has several stages. Each stage comprises of taking a data set, using Python scripts, auxiliary data, configuration and parameters, and ...
45
votes
9answers
60k views

IDE alternatives for R programming (RStudio, IntelliJ IDEA, Eclipse, Visual Studio)

I use RStudio for R programming. I remember about solid IDE-s from other technology stacks, like Visual Studio or Eclipse. I have two questions: What other IDE-s than RStudio are used (please ...
37
votes
11answers
23k views

Do data scientists use Excel?

I would consider myself a journeyman data scientist. Like most (I think), I made my first charts and did my first aggregations in high school and college, using Excel. As I went through college, ...
24
votes
5answers
3k views

VM image for data science projects

As there are numerous tools available for data science tasks, and it's cumbersome to install everything and build up a perfect system. Is there a Linux/Mac OS image with Python, R and other open-...
23
votes
4answers
9k views

What makes columnar databases suitable for data science?

What are some of the advantages of columnar data-stores which make them more suitable for data science and analytics?
15
votes
2answers
3k views

What is the difference between Hadoop and noSQL

I heard about many tools / frameworks for helping people to process their data (big data environment). One is called Hadoop and the other is the noSQL concept. What is the difference in point of ...
11
votes
3answers
4k views

Best languages for scientific computing [closed]

It seems as though most languages have some number of scientific computing libraries available. Python has Scipy Rust has <...
11
votes
2answers
110 views

Opensource tools for help in mining stream of leader board scores

Consider a stream containing tuples (user, new_score) representing users' scores in an online game. The stream could have 100-1,000 new elements per second. The ...
10
votes
4answers
764 views

What initial steps should I use to make sense of large data sets, and what tools should I use?

Caveat: I am a complete beginner when it comes to machine learning, but eager to learn. I have a large dataset and I'm trying to find pattern in it. There may / may not be correlation across the data,...
10
votes
2answers
329 views

Book keeping of experiment runs and results

I am a hands on researcher and I like testing out viable solutions, so I tend to run a lot of experiments. For example, if I am calculating a similarity score between documents, I might want to try ...
8
votes
3answers
499 views

Google prediction API: What training/prediction methods Google Prediction API employs?

The details of the Google Prediction API are on this page, but I am not able to find any details about the prediction algorithms running behind the API. So far I have gathered that they let you ...
7
votes
4answers
3k views

What are helpful annotation tools (if any)

I'm looking for tools that would help me and my team annotate training sets. I work in an environment with large sets of data, some of which are un- or semi-structured. In many cases there are ...
6
votes
2answers
596 views

Running an R script programmatically [closed]

I have an R script that generates a report based on the current contents of a database. This database is constantly in flux with records being added/deleted many times each day. How can I ask my ...
6
votes
2answers
378 views

Is Keras a fancy tool or is it actually helpful in research or both?

I’m in second year of my PhD in a environmental-science laboratory and there aren’t many experts on machine learning here. In one part of my research I should use a multilayer perceptron to build a ...
6
votes
1answer
1k views

How to place XGBoost in a full stack for ML?

Is XGBoost complete by itself for prod-strength machine learning? If not, with which other tools or libs is it typically combined, and how? (I recently read a description of a stack that included ca ...
6
votes
3answers
1k views

Summarize and visualize a CSV in Java/Scala?

I would like to summarize (as in R) the contents of a CSV (possibly after loading it, or storing it somewhere, that's not a problem). The summary should contain the quartiles, mean, median, min and ...
6
votes
1answer
169 views

Which Big Data technology stack is most suitable for processing tweets, extracting/expanding URLs and pushing (only) new links into 3rd party system?

(Note: Pulled this question from the list of questions in Area51, but believe the question is self explanatory. That said, believe I get the general intent of the question, and as a result likely able ...
5
votes
2answers
554 views

Do you need a virtual machine for data science?

I am brand new to the field of data science, want to break into it, and there are so many tools out there. These VMs have a lot of software on them, but I haven't been able to find any side-by-side ...
5
votes
1answer
667 views

Lightweight data provenance tool

One of the problems I often encounter is that of poor data provenance. When I do research I continuously make modifications to my code and rerun experiments. Each time I'm faced with a number of ...
4
votes
2answers
983 views

Does anyone use Julia programming language? [closed]

Is anyone using Julia (http://julialang.org/) for professional jobs? Or using it instead of R, Matlab, or Mathematica? Is it a good language? If you have to ...
3
votes
2answers
1k views

Tool to Generate 2D Data via Mouse Clicking

Often when I am learning new machine learning methods or experimenting with a data analysis algorithm I need to generate a series of 2D points. Teachers also do this often when making a lesson or ...
3
votes
2answers
951 views

Cheat Sheet of UNIX commands for Data Science

I was looking for a cheat sheet of UNIX commands, which are specifically usable for data science. I mean an introduction to very basic commands (starting from cd, <...
3
votes
2answers
10k views

List of used aliases in Tableau

I often get the problem when this or that alias name is already used somewhere, and I can't easily find that variable or aggregation to release the name. Is there some place in Tableau where I can ...
3
votes
1answer
61 views

What's the difference between data products and intelligent systems?

Basically, both are software systems that are based on data and algorithms.
2
votes
2answers
564 views

Why has Hadoop failed to become popular?

Apache Hadoop was once considered one of the tools you should have as a Data Scientist. Around 2012 to 2014, it gained the popularity steeply among Data Scientists, and seemed to be considered one of ...
2
votes
3answers
1k views

Is there any book for modern optimization in Python?

I was reading Modern Optimization with R (Use R!) and wondering if a book like this exists in Python too? To be precise something that covers stochastic gradient descent and other advanced ...
2
votes
2answers
500 views

Is there any augmentation tool for images and bounding boxes?

I don't have a lot of training data and I'm looking for some tools in python or executable program like labelimg that do some heavy augmentation on images, even better if they also change bounding ...
2
votes
1answer
138 views

Using doccano for Aspect Based Sentiment Analysis annotation

Currently looking for a good tool to annotate sentences regarding aspects and their respective sentiment polarities. I'm using SemEval Task 4 as a reference. The following is an example in the ...
2
votes
1answer
31 views

Tool for clustering and cleansing data set

I have a large-ish data set (400K records) composed of two fields (both strings). I am looking for a tool that will enable me to cluster the data e.g. around the first column, either using exact ...
2
votes
0answers
111 views

Linux tools to handle large data sets (several TB) [closed]

I come from a physics background that has a very specific tool (ROOT, a C++ toolkit) to handle very large data sets for a very specific purpose. For smaller data sets (MB to GB), I use modules in ...
2
votes
0answers
11 views

Reusable parameter scans wrapper

In most of my projects, I come up with models and want to visualize how some property $x$ varies as a function of a subset of parameters $p_1$,$p_2$, .. etc. So I'll often end up with figures of the "...
2
votes
0answers
220 views

Making Factual drake work on Windows 7 64-bit

I have installed Drake on Windows 7 64-bit. I am using JDK 1.7.0_51. I tried both using the pre-compiled jar file and compiling from the Clojure source using leiningen. The resulting Drake version ...
1
vote
1answer
702 views

Any clear winner for Data Science in Scala? [closed]

I have found a number of libraries and tools for data science in Scala, I would like to know about which one has more adoption and which one is gaining adoption at a faster pace and to what extent ...
1
vote
1answer
807 views

Best tool for text pre-processing, involving tokenization, lemmatization, stop-word removal, feature vector extraction?

For text processing there are plenty of tools out there like CoreNLP, SpaCy, NLTK, textblob etc. each offering different suites of pre-processing functions and people recommend different tool for ...
1
vote
1answer
38 views

Are deep - learning toolkits targeted for certain areas or all-purpose tool kits?

Are any of the open deep learning toolkits targeted to certain areas, or a all toolkits all purpose toolkits, meaning it is a blackbox for deep learning. My question comes in regards to Microsoft's ...
1
vote
1answer
46 views

Data Exploration Tips & Tools

I am pretty new to the Data Science world and I am looking for some advices. One of the tasks that I have to do, is to go alone at a customer place for few days/weeks and dig into their data lake to ...
1
vote
1answer
406 views

Tool for designing CNN architectures

I am currently building a (simple sequential) fully convolutional network for an object recognition task. Designing the architecture essentially amounts to choosing kernel size, stride and unit ...
1
vote
2answers
114 views

Visualize changes other time

I am analyzing a population movement pattern and I would like to design a visualization like this one: Is there any tool or language (preferably R) I can use for that purpose? Here is a sample of my ...
1
vote
1answer
73 views

Query related to Matlab Neural Network toolbox

If anyone has used the Neural Network toolbox in matlab, what does the two values in the performance section means? Like considering this image: The performance section has the value 0.484 and 1.32e-...
1
vote
0answers
28 views

App or Service to QA 80,000 classified images?

I've been reviewing services and apps for the past week and my head is exploding. I just haven't found the right one yet, but I know it has to be out there, so I'm turning to the collective wisdom ...
1
vote
0answers
17 views

Any Hadoop Tool to Annotate Text

I have large text files on HDFS... I would like to label some text in those files to improve text analysis? Do you know of any tool like that?
1
vote
0answers
90 views

How to find relationships in datasets and predict data

I'm a .NET developer trying to learn more about AI/machine learning. I have a dataset with few variables and known results. I'm looking for a language/framework/tool/approach to analyse the data and ...
1
vote
0answers
16 views

Run queries against results of previously cached queries [closed]

I'm trying to find a tool or workflow that would allow a team to analysis data in multiple steps, where intermediate results are cached. (I'll assume that the raw data is in a relational database and ...
1
vote
1answer
61 views

How to extract most occuring words based on month & what tool to use?

Hi guys I'm very new to data science, I have intermediate background on programming and have used Pentaho Data Integration tool once for DB migration & data cleansing. Let's say I have this kind ...
1
vote
0answers
153 views

Tool for visualizing long time series data

I am searching for a software / tool / R package (any of these) that enables easy visualization of a time series. My problem is the following: I have a multiple time series with 7 attributes I want ...
1
vote
0answers
44 views

Is there a machine learning framework that supports partial evaluation ie can return a function?

Is there a machine learning framework that supports partial evaluation? For example: We train on [model, year, km, ..., colour, price]. Today we call ...
1
vote
0answers
18 views

What tools are out there to collect participants' browsing and/or search data as part of an experiment?

I'm running an experiment where I need to collect and analyse participants' browsing and search histories. The design of the experiment is similar to an "instrumented user panel", described here:http:/...
0
votes
1answer
4k views

Do Data Scientists prefer MACS? [closed]

I see many Data Science (DS) tutorials done on MACS, and many DS blogs recommend MACS as the best developing platform, thus the quote "Data Science is statistics on a Mac" came more than once into my ...
0
votes
0answers
3 views

Are there any tools, packages or frameworks that are used to record model parameters, the number of kfols, local cv and so on?

I have been trying to do some experiments or data competitions but I always try to record the parameters of the model, the number of kfolds, local cv and so on. So I am wondering if there is some ...