Questions tagged [tools]
Use for questions about the different software and hardware used to assist in, and solve, data science problems
70 questions
0
votes
1
answer
15
views
Managing the convenience of downloading data to your laptop VS the security risk of having data in a laptop that is less secure than the cloud
As datascientists and ML engineers, we have often pulled data into our laptops to explore and even to prototype models. But laptops are are not secure as services within the company's cloud, and this ...
- 101
0
votes
1
answer
23
views
Interactive image segmentation training tool
I have a bunch of 2D grayscale images for which I want to train a (multi-label) segmentation model. What is the simplest way to interactively train such a model?
I.e., I want to:
Label a (very) few ...
- 111
0
votes
1
answer
23
views
Searching ChatGPT transcripts?
Maybe I just don't understand the functionality of the web version. Using GPT4 here.
When I do technical queries (usually TensorFlow applications) I use similar prompts until I get the understanding I ...
- 33
0
votes
0
answers
76
views
Preparation for data scientist interviews/job for final year PhD student in ML theory
I would like to check/ask if there is anything particular that I need to prepare for data scientist interviews. I am quite unsure/lost about the requirements since I am coming from academia (see my ...
- 101
0
votes
1
answer
162
views
Is it possible to use the SODA tool as open source?
I was reading about the SODA tool and was wondering if it was free to use. As far as I understand, it has a SODA library and a SODA core, but they are all just clients for connecting to a cloud server ...
- 1
5
votes
2
answers
1k
views
How to handle similarity search on mixed data types vectors?
I think this question is one that many beginners run into and I could not find a decent generic guide for it.
My issue is the following. I want to evaluate similarity of vectors which have mixed data ...
- 53
3
votes
3
answers
264
views
What's your ideal work environment?
I'm a founder in a data science heavy startup, and I'm currently functioning as the entire dev team. Before I know it we'll have people working together on a project I, currently, work on completely ...
- 430
1
vote
0
answers
151
views
How to integrate Zoho analytics with Jupyter notebook? [closed]
I am trying to connect Zoho Analytics and Python for importing data from Zoho Analytics.
I have already installed !pip install zoho-analytics-connector. What should ...
- 11
1
vote
0
answers
22
views
Cloud-based visual tool to perform NLP on text corpora
I have some text corpora to share with non-programming clients (~50K documents, ~100M tokens) who would like to perform operations like regex searches, co-locations, Named-entity recognition, and word ...
- 173
1
vote
0
answers
21
views
UI-based Tool for Qualitative Evaluation of Data Quality
Dear DS StackExchange community,
I'm currently searching the interwebs for a (near-)ready-to-use solution, to perform a qualitative evaluation of extracted features from video data. In my head the ...
0
votes
1
answer
43
views
Data science tools for easing the participation of a business into their scoring system
I'm a working in a small company. The company sells products on a website and they have a python script that runs everyday to attribute a score to each product based on a set of parameters (google ...
- 103
1
vote
0
answers
34
views
Suggestions for Open-Source Tool for Image Classifications (with Nesting)
I'm looking for an open source tool to assist my colleagues and I to label images for a machine learning application. We don't actually need bounding boxes or anything to pinpoint regions within each ...
- 111
2
votes
1
answer
115
views
What is the difference between Pachyderm and Git?
I learned that tools like Pachyderm version-control data, but I cannot see any difference between that tool with Git. I learned from this post that:
It holds all your data in a central accessible ...
- 526
4
votes
2
answers
140
views
Tools for project management Data Science
I'm in charge of a small data science team (3 data scientists, me included). We do our projects with at least one business person (PM) per project ( we have 5 of these).
We managed everything with ...
- 191
2
votes
1
answer
3k
views
How to change data source names in Google Data Studio
Google Data Studio has connectors to MySQL, PostgreSQL, etc. but these connections will come with default names.
Couldn't find out how to set the names for data sources in Google Data Studio. Is it ...
- 125
4
votes
2
answers
131
views
Data Engineering Stack - collect, transform and visualize geospatial data
I'm making a side project, where I collect geospatial data by web scrapping and from OSM API. I've started with simple Java application, however, I would like to make it as a data flow, purely for ...
- 141
2
votes
0
answers
22
views
Tools/tutorials for compiling corpora for NLP experiments? [closed]
I have a couple of NLP ideas I want to try out (mostly for my own learning) - while I have the python/tensorflow background for running the actual training and prediction tasks, I don't have much ...
- 487
1
vote
0
answers
37
views
What VM should I use? [closed]
So I am starting a project that has some code in it that has to do with machine learning (for example recommender system).
At the moment I am thinking that if I want to deploy this application, should ...
- 11
1
vote
1
answer
20
views
Is there a way to test out simple filters before committing to coding them?
Is there a way to test out simple filters before committing to coding them?
Like if I want to estimate the feasibility of recognizing some features from images. Or to estimate the effort/...
- 416
1
vote
0
answers
41
views
App or Service to QA 80,000 classified images?
I've been reviewing services and apps for the past week and my head is exploding. I just haven't found the right one yet, but I know it has to be out there, so I'm turning to the collective wisdom ...
- 11
4
votes
2
answers
587
views
Using doccano for Aspect Based Sentiment Analysis annotation
Currently looking for a good tool to annotate sentences regarding aspects and their respective sentiment polarities.
I'm using SemEval Task 4 as a reference. The following is an example in the ...
- 153
3
votes
2
answers
101
views
Tool for clustering and cleansing data set
I have a large-ish data set (400K records) composed of two fields (both strings). I am looking for a tool that will enable me to cluster the data e.g. around the first column, either using exact ...
- 139
6
votes
2
answers
825
views
Is Keras a fancy tool or is it actually helpful in research or both?
I’m in second year of my PhD in a environmental-science laboratory and there aren’t many experts on machine learning here.
In one part of my research I should use a multilayer perceptron to build a ...
Andrew
2
votes
4
answers
4k
views
Is there any augmentation tool for images and bounding boxes?
I don't have a lot of training data and I'm looking for some tools in python or executable program like labelimg that do some heavy augmentation on images, even better if they also change bounding ...
0
votes
1
answer
8k
views
Do Data Scientists prefer MACS? [closed]
I see many Data Science (DS) tutorials done on MACS, and many DS blogs recommend MACS as the best developing platform, thus the quote "Data Science is statistics on a Mac" came more than once into my ...
- 161
0
votes
2
answers
45
views
Keep track of trainings, datasets eetc
After searching quite some time for it on Google I could not find a sufficient software/toolbox that can manage trainings of neural networks. I thought of a program that combines visualization ...
- 11
1
vote
0
answers
22
views
Any Hadoop Tool to Annotate Text
I have large text files on HDFS... I would like to label some text in those files to improve text analysis?
Do you know of any tool like that?
13
votes
5
answers
12k
views
What are helpful annotation tools (if any)
I'm looking for tools that would help me and my team annotate training sets. I work in an environment with large sets of data, some of which are un- or semi-structured. In many cases there are ...
1
vote
0
answers
128
views
Finding relationships in datasets and predicting data
I'm a .NET developer trying to learn more about AI/Machine Learning.
I have a dataset with few variables and known results. I'm looking for a language/framework/tool/approach to analyze the data and ...
- 111
2
votes
1
answer
875
views
Tool for designing CNN architectures
I am currently building a (simple sequential) fully convolutional network for an object recognition task.
Designing the architecture essentially amounts to choosing kernel size, stride and unit ...
- 121
2
votes
2
answers
673
views
Why has Hadoop failed to become popular?
Apache Hadoop was once considered one of the tools you should have as a Data Scientist. Around 2012 to 2014, it gained the popularity steeply among Data Scientists, and seemed to be considered one of ...
- 911
1
vote
1
answer
41
views
Are deep - learning toolkits targeted for certain areas or all-purpose tool kits?
Are any of the open deep learning toolkits targeted to certain areas, or a all toolkits all purpose toolkits, meaning it is a blackbox for deep learning.
My question comes in regards to Microsoft's ...
- 619
1
vote
0
answers
18
views
Run queries against results of previously cached queries [closed]
I'm trying to find a tool or workflow that would allow a team to analysis data in multiple steps, where intermediate results are cached. (I'll assume that the raw data is in a relational database and ...
- 111
1
vote
1
answer
75
views
How to extract most occuring words based on month & what tool to use?
Hi guys I'm very new to data science,
I have intermediate background on programming and have used Pentaho Data Integration tool once for DB migration & data cleansing.
Let's say I have this kind ...
- 13
1
vote
2
answers
214
views
Visualize changes other time
I am analyzing a population movement pattern and I would like to design a visualization like this one:
Is there any tool or language (preferably R) I can use for that purpose?
Here is a sample of my ...
- 55
2
votes
1
answer
917
views
Best tool for text pre-processing, involving tokenization, lemmatization, stop-word removal, feature vector extraction?
For text processing there are plenty of tools out there like CoreNLP, SpaCy, NLTK, textblob etc. each offering different suites of pre-processing functions and people recommend different tool for ...
- 3,845
2
votes
0
answers
175
views
Tool for visualizing long time series data
I am searching for a software / tool / R package (any of these) that enables easy visualization of a time series.
My problem is the following:
I have a multiple time series with 7 attributes
I want ...
- 113
1
vote
0
answers
99
views
Is there a machine learning framework that supports partial evaluation ie can return a function?
Is there a machine learning framework that supports partial evaluation?
For example:
We train on [model, year, km, ..., colour, price].
Today we call
...
1
vote
1
answer
29
views
What tools are out there to collect participants' browsing and/or search data as part of an experiment?
I'm running an experiment where I need to collect and analyse participants' browsing and search histories. The design of the experiment is similar to an "instrumented user panel", described here:http:/...
- 111
3
votes
5
answers
5k
views
Is there any book for modern optimization in Python?
I was reading Modern Optimization with R (Use R!) and wondering if a book like this exists in Python too? To be precise something that covers stochastic gradient descent and other advanced ...
- 865
57
votes
8
answers
14k
views
Why do internet companies prefer Java/Python for data scientist job?
I see a many times in job description for data scientist asking for Python/Java experience and disregard R. Below is a personal email I received from chief data scientist of a company I applied for ...
- 865
2
votes
0
answers
168
views
Linux tools to handle large data sets (several TB) [closed]
I come from a physics background that has a very specific tool (ROOT, a C++ toolkit) to handle very large data sets for a very specific purpose. For smaller data sets (MB to GB), I use modules in ...
- 245
2
votes
1
answer
23
views
Reusable parameter scans wrapper
In most of my projects, I come up with models and want to visualize how some property $x$ varies as a function of a subset of parameters $p_1$,$p_2$, .. etc.
So I'll often end up with figures of the "...
- 121
6
votes
1
answer
1k
views
How to place XGBoost in a full stack for ML?
Is XGBoost complete by itself for prod-strength machine learning? If not, with which other tools or libs is it typically combined, and how?
(I recently read a description of a stack that included ca ...
10
votes
5
answers
4k
views
Tool to Generate 2D Data via Mouse Clicking
Often when I am learning new machine learning methods or experimenting with a data analysis algorithm I need to generate a series of 2D points. Teachers also do this often when making a lesson or ...
- 310
1
vote
1
answer
168
views
Query related to Matlab Neural Network toolbox
If anyone has used the Neural Network toolbox in matlab, what does the two values in the performance section means? Like considering this image:
The performance section has the value 0.484 and 1.32e-...
- 1,161
28
votes
4
answers
11k
views
What makes columnar databases suitable for data science?
What are some of the advantages of columnar data-stores which make them more suitable for data science and analytics?
- 8,416
4
votes
2
answers
1k
views
Cheat Sheet of UNIX commands for Data Science [closed]
I was looking for a cheat sheet of UNIX commands, which are specifically usable for data science. I mean an introduction to very basic commands (starting from cd, <...
- 766
7
votes
1
answer
780
views
Lightweight data provenance tool
One of the problems I often encounter is that of poor data provenance.
When I do research I continuously make modifications to my code and rerun experiments. Each time I'm faced with a number of ...
- 245
43
votes
10
answers
33k
views
Do data scientists use Excel?
I would consider myself a journeyman data scientist. Like most (I think), I made my first charts and did my first aggregations in high school and college, using Excel. As I went through college, ...
- 533