Questions tagged [databases]

A comprehensive collection of related data organized for convenient access, generally associated with software to update and query the data.

Filter by
Sorted by
Tagged with
2
votes
1answer
913 views

word2vec -storing a word and its vector yet be able to efficiently run k-nearest neighbour

After training a model using word2vec I'd now like to store the trained model with the word serving as a key and the vector as its value. However I'm not sure how I'll be able to implement in this way ...
0
votes
3answers
3k views

@RISK Vs R. Monte Carlo simulation in R?

Are there any cheaper or open source alternatives to @RISK or are there packages for R that would be able to perform the same tasks? http://www.palisade.com/risk/ @RISK (pronounced “at risk”) ...
1
vote
1answer
1k views

Implementing Frequently bought together using a DB

We have a classic structure of an online shop database (products, customers, sales) and we want to implement a Frequently bought together feature. Our software is in ASP.NET and we do not know PHP to ...
1
vote
1answer
778 views

Overcome memory limitation when downloading from database into Orange

I would like to run the association rule mining algorithm of the Orange library on a dataset that is stored in a PostgreSQL database. The table 'buildingset' contains the itemsets for each user, thus ...
2
votes
1answer
274 views

NoSQL vs SQL backend for semi structured data

I have a corpus of job descriptions and another corpus of CVs of applicants. I plan to implement a matching system using machine learning algorithms, to find top 5 or top 10 applicants for each job ...
2
votes
0answers
2k views

Decode SQLite database blobs: how to start?

Updated: see bottom of this question. Not sure if I am at the right channel here but I have good hopes someone here might be able to help me. I am trying to process and analyse data from a Footscan ...
2
votes
2answers
329 views

Data structure design for supporting arbitrary number of columns in table or database

I'm currently working on a sort-of a meta-modeler to build a free web service so that people can input data and run several models on that data. The task I'm currently struggling is: user needs to ...
25
votes
4answers
9k views

What makes columnar databases suitable for data science?

What are some of the advantages of columnar data-stores which make them more suitable for data science and analytics?
0
votes
1answer
61 views

Looking for a 'CITY, STATE' within a body of text (from a CITY-STATE database)

I'm looking for an optimal way to search a large body of text for a combination of words that resemble any CITY, STATE combination I have in a separate CITY-STATE database. My only idea would be to ...
2
votes
3answers
157 views

Can Hadoop be beneficial when data is in database tables and not in a file system

I work for a bank. Most of our data is in the form of database tables. Would we benefit by implementing Hadoop? I am of the impression that Hadoop is more for a Distributed File System (unstructured ...
1
vote
2answers
55 views

Merging two datasets with unclear identifier in Excel

I'm trying to merge two datasets on Mergers & Acquisitions. They both consist of c.10'000 observations with c.50-100 variables each. One contains information about the actual M&A deal whereas ...
1
vote
2answers
777 views

any reason for this project to use hadoop/spark?

I am setting up for a couple self study projects to explore machine learning techniques. 1st project has 10,000 time series with 24 float data points each day for 10 years (876 million points). I ...
5
votes
3answers
848 views

Storing Sensor Data for Analysis of the Office

I have currently been tasked with designing an application that tracks several different measurements around the office, eg. the temperature, light, presence of people, etc. Having never really worked ...
2
votes
2answers
148 views

Data repositories like UCI

Are there any other data repositories like UCI and mlData, for biological data?? I want to know about mostly biological data set.
10
votes
3answers
7k views

Which is faster: PostgreSQL vs MongoDB on large JSON datasets?

I have a large dataset with 9m JSON objects at ~300 bytes each. They are posts from a link aggregator: basically links (a URL, title and author id) and comments (text and author ID) + metadata. They ...
2
votes
2answers
184 views

How to make an effective sampling from a database of text documents?

Problem: I want to know methods to perform an effective sampling from a database. The size of the database is about 250K text documents and in this case each text ...
48
votes
9answers
21k views

How to deal with version control of large amounts of (binary) data

I am a PhD student of Geophysics and work with large amounts of image data (hundreds of GB, tens of thousands of files). I know svn and ...
13
votes
1answer
7k views

Neo4j vs OrientDB vs Titan

I am working on a data-science project related on social relationship mining and need to store data in some graph databases. Initially I chose Neo4j as the database. But it seams Neo4j doesn't scale ...
1
vote
0answers
746 views

R-Cassandra connection with RJDBC

I am trying to connect with cassandra in R using RJDBC. When I execute casscon <- dbConnect(cassdrv, "jdbc:cassandra://ipaddrs:9160/demodb") I am getting <...
6
votes
2answers
3k views

Python interface to Titan Database

How can I connect to Titan database from Python ? What I understand is that Titan (Graph database) provides an interface (Blueprint) to Cassandra (Column Store) and bulb is a python interface to ...
4
votes
1answer
2k views

Database for a trie, or other appropriate structure for recommendation engine

We are storing the information about our users showing interest in our items. Based on this information, we would like to create a simple recommendation engine that will take the items I1, I2, I3 etc ...
5
votes
1answer
164 views

is this a good case for NOSQL?

I'm currently facing a project that I could solve with a relational database in a relatively painful way. Having heard so much about NOSQL, I'm wondering if there is not a more appropriate way of ...
12
votes
3answers
3k views

Efficient database model for storing data indexed by n-grams

I'm working on an application which requires creating a very large database of n-grams that exist in a large text corpus. I need three efficient operation types: Lookup and insertion indexed by the n-...
9
votes
3answers
4k views

Human activity recognition using smartphone data set problem

I'm new to this community and hopefully my question will well fit in here. As part of my undergraduate data analytics course I have choose to do the project on human activity recognition using ...
6
votes
1answer
531 views

Is Data Science just a trend or is a long term concept? [closed]

I see a lot of courses in Data Science emerging in the last 2 years. Even big universities like Stanford and Columbia offers MS specifically in Data Science. But as long as I see, it looks like data ...
13
votes
1answer
216 views

When a relational database has better performance than a no relational

When a relational database, like MySQL, has better performance than a no relational, like MongoDB? I saw a question on Quora other day, about why Quora still uses MySQL as their backend, and that ...
10
votes
2answers
923 views

Is this Neo4j comparison to RDBMS execution time correct?

Background: Following is from the book Graph Databases, which covers a performance test mentioned in the book Neo4j in Action: Relationships in a graph naturally form paths. Querying, or ...
5
votes
2answers
626 views

Running an R script programmatically [closed]

I have an R script that generates a report based on the current contents of a database. This database is constantly in flux with records being added/deleted many times each day. How can I ask my ...
2
votes
0answers
647 views

What are the advantages and disadvantages of SQL versus NoSQL in data science? [closed]

In which situations would one system be preferred over the other? What are the relative advantages and disadvantages of relational databases versus non-relational databases?

1
2