Questions tagged [sql]

Structured Query Language (SQL) is a language for querying databases. Questions should include code examples, table structure, sample data, and a tag for the DBMS implementation (e.g. MySQL, PostgreSQL, Oracle, MS SQL Server, IBM DB2, etc.) being used.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

SQL Server Management Studio for Ubuntu alternative [migrated]

I am looking for an alternative to SQL Server Management Studio for Ubuntu. I have worked with Pycharm Pro so far but I currently don't have the pro version. I looking for a noob friendly and with ...
0
votes
0answers
12 views

Scraping mixed elements and passing to SQL

I'm running webscrapes via python which are retrieving data from csv's hosted on the web. I'd like to pass the data into a MSSQL database. An issue I have is the mixed elements/data types in the csv....
1
vote
1answer
50 views

How to plot time series data from SQLite database

I am trying to plot time series data by accessing the SQLite database. I successfully connect the database by coding the below script,however, i couldn't figure out how to plot the time series data ...
0
votes
1answer
38 views

SQL and JSON database - R

I have a database (PostgresSQL) which looks like this: ""results"": {""WEBS"": {""WEBS_4"": [{""ip"": ""XXXXX"", ""value"": ""YYYY""}]}} I would ...
0
votes
0answers
17 views

How to store subset of columns from a csv file?

I need to create a table in hive (or Impala) by reading from a csv file (named file.csv), ...
0
votes
0answers
13 views

Minimal Fragmentation in Distributed Database

In a distributed database, using selection we get a minimal fragmentation if If all the predicates of a set Pr are relevant, Pr is minimal. Given fragments generated by a set $M(P)$, we say a ...
0
votes
0answers
16 views

how to run pyspark on Hadoop cluster, cache data, then read file using pandas on local machine via its URI?

I connect to a Hadoop cluster via vpn. I want to execute a script that makes sql call and saves the data to a file on the remote machine, then using its URI I want to read on my local running Jupiter ...
0
votes
0answers
35 views

Self Union All in Tableau Public for Sankey Diagram

I'm following this 2017 youtube tutorial from-which I hope to learn Sankey diagrams. At 2:28 he instructs us to select Convert to Custom SQL, after having opened ...
0
votes
0answers
15 views

What are best practice ETL stages for an API to SQL datamart?

Here's the scenario: I would like to create a datamart for US patents using the PatentsView JSON API. This requires dumping a large amount of data once and then appending thereafter in regular ...
0
votes
1answer
37 views

How to execute a git project with different input data?

I would like to run this project and receive result from this project However I have problem to execute the step mailcorpus.json created by executing the sql script data/get_mail_corpus.sql on ...
0
votes
0answers
29 views

Prepping Sales data for Basket Analysis

I've been playing around trying to teach my self some basic data science with datasets I have access to work, I've just finished prepping a 46 million row dataset which I wrangled using pandas and sql ...
0
votes
0answers
15 views

Power BI RLS through SQL stored procedure

I need dynamic RLS for Power BI - the way that the client has their data structured is that they have SQL RLS through a stored procedure. How can I translate that into power bi?
0
votes
1answer
17 views

CASE sentence for SQL query

I have a question abut MySQL I would like to handle a column with characters without changing the original DB. Below is the levels of the column named 'Maker'. 기아, 현대, BMW, VolksWagen, Porsche So ...
1
vote
1answer
232 views

What should I master better for professional data science in economics and finance? [closed]

First, excuse me for the noob and long question which is probably doesn’t even belong to here, I know there are several question been answered like this out there, but I think this is going to be up-...
2
votes
2answers
82 views

How to deal with millions or rows of data for analysis/visualization purpose [closed]

I have data in 2 tables in Sql server. First table has around 10 million rows and 8 columns. Second table has 6 million rows and 60 columns. I want to import those tables into a Python notebook ...
5
votes
1answer
41 views

Data enrichment of geographical records

I have a user_data table with various fields, some of them are based on geography. I'd like to enrich the data with additional columns, like ...
4
votes
4answers
68 views

How do Data Scientists integrate Predictive Modeling with SQL?

I've heard a lot about the importance of knowing SQL but how do you guys at the larger firms integrate your software for modeling/analysis with the server that holds the data? Do you simply access ...
0
votes
0answers
6 views

Create a local subset copy of a cloud based MS SQL database

I have a large MS SQL database hosted on Azure. I want to develop an analysis pipeline for the data. I'm developing this pipeline locally and it's slowing down my development. I think if I had a local ...
0
votes
0answers
16 views

Analyze additions and subtractions to/from a data set

I have a data set in the following form: ...
0
votes
0answers
29 views

Filtering unique row values in SQL, Advanced

My data looks like this: Number(String), Number2(String), TransactionType(String), Cost(Integer) For number 1, Cost 10 and -10 cancel out so the remaining cost is 100 For number 2, Cost 50 and -...
1
vote
0answers
20 views

Probabilistic guarantees for SQL query execution

We are working on a project that tries to accelerate Spark SQL using dedicated hardware. Our approach works much better (in terms of the resulting latency) if we allow query processing that is only ...
1
vote
1answer
35 views

find closest numbers approximately

I have a table like this: TableTest Col1 Col2 Col3 Col4 5 6 7 8 12 6 5 6 2 3.5 6 1 And I want to find the closest row ...
0
votes
1answer
744 views

Python & Pandas : (pymysql.err.OperationalError) (2003, “Can't connect to MySQL server on 'localhost' (timed out)”)

I'm writing a Python Script to store JSON data into MySQL Database. I used pandas to store into MySQL Database. I used two different modules (...
3
votes
1answer
2k views

Python & Pandas : TypeError: to_sql() got an unexpected keyword argument 'flavor'

I want to store JSON Data into MySQL Database using Python. I used dataframe of pandas. I found that ...
3
votes
1answer
44 views

How to find the values that never matched in a full join

Hello I have done this full join query: ...
1
vote
0answers
16 views

Qualitative prediction based off data containing mixed qualitative and quantitative values? [closed]

I'm trying to find a way to predict an integer value based off of an item's prior sale history and am inquiring for a starting point on how to approach this. The factors are as follows (Data Type in ...
3
votes
2answers
74 views

Mean across every several rows in pandas

I have a table of features and labels where each row has a time stamp. Labels are categorical. They go in a batch where one label repeats several times. Batches with the same label do not have a ...
0
votes
1answer
99 views

What algorithms can be used to derive matching rules between known matches in datasets?

Lets say I have two datasets with different column names except for a unique ID key Table 1 CSV ...
0
votes
1answer
211 views

What algorithm could be used to fuzzy merge multiple datasets?

Problem Description I have several tables that are related but do not share any unique key. I've come across this problem several times with customer data in separate source systems that needs to be ...
3
votes
1answer
1k views

What is the best way to read SQL dataset in to Tensorflow?

What is the best way to read SQL database in to Tensorflow? Currently, I am using Postgres on server and developed DL algorithm on Tensorflow using Jupyter Lab. How can I import data into Jupyter Lab ...
1
vote
0answers
232 views

Solving correlation and covariance with MySQL

I am new to data science taking my first course in Data analysis. I am trying to figure out where to start with a problem solving the correlation and covariance with my sql. ...
1
vote
0answers
205 views

How to pass database name as parameter to sql statement in python? [closed]

I have several SQL statements (update, delete, insert,...) in my Python code and all of them ...
74
votes
11answers
41k views

Why do people prefer Pandas to SQL?

I've been using SQL since 1996, so I may be biased. I've used MySQL and SQLite 3 extensively, but have also used Microsoft SQL Server and Oracle. The vast majority of the operations I've seen done ...
1
vote
0answers
111 views

Python-SQL database [closed]

There are two SQL databases in which one database contains user data and other database contains service data. Firstly, I am connecting the databases and fetching it on python.Based on the user data, ...
0
votes
1answer
191 views

How to use features when predicting aggregation?

Let's say I have to predict the total monthly sales of a store. I have the data in the following format: I don't think I can use this data directly for predicting the sales for a month, as in it's ...
1
vote
1answer
43 views

Call a SQL procedure, for each group by [closed]

I am trying to call a procedure for each group of data. For example, I have a list of customers and movie's features that they like. Is a relation one to many, I have many records for one specific ...
1
vote
1answer
234 views

Is there an analog to SQL's STRING_AGG (or FOR XML PATH) function in Python?

Asked this in SE but maybe this is too data-oriented so trying to post it here. I am trying to find the analog to the SQL function STRING_AGG so that I can concatenate across columns and rows of a ...
11
votes
3answers
17k views

Natural Language to SQL query

I have been working on developing a system "Converting Natural Language to SQL Query". I have read the answers from the similar questions, but was not able to get the information that I was looking ...
1
vote
0answers
64 views

How to Return Mean Response Values using dplyr and SQL Server R Services?

I'm working on a stored procedure in SQL Server that utilizes dplyr (which is installed) to obtain the mean of response values for the responses to a given study ID. The R script and stored procedure ...
1
vote
1answer
446 views

Most Efficient Post Processing with Python and Pandas

This question is about best practices for working in Pandas dataframes. Speed, ease of use, and memory consumed could all impact any answers you might have. I start by pulling a data set into a ...
4
votes
3answers
525 views

The right join for the right situation?

(Please pardon the pun in the question title.) I'm currently teaching an introductory class to data science. I understand the differences between a left join, a right join, an inner join and a full ...
2
votes
2answers
3k views

Is there a package for using SQL to manipulate Pandas dataframes in Python?

Rather than learn a new package/language, I'd like to use my existing SQL skills to manipulate pandas dataframes in Python. Does anyone know of a way to do this, or ...
-1
votes
1answer
149 views

Tableau: How to change multiple field names

From SQL server I imported multiple tables that each have multiple fields. Unfortunately the field names are not that descriptive (data is originally from SAP) but I have a separate Excel file that ...
4
votes
2answers
117 views

Seeking advice on database architecture — given my problem, what tools should I learn?

I'm a fairly experienced R user, but until now I haven't had a good reason to learn to use databases. Now I have a problem where I am dealing with model output that I need to save to disk, and then ...
1
vote
1answer
284 views

Data dashboard for SQL server database

I am working on developing a data dashboard / app for a Microsoft SQL database. Currently, I am developing the dashboard using Shiny and R. The app is mostly for exploratory analysis to allow people ...
-1
votes
1answer
2k views

Spark: calculating percentage from a non-Int field

I have a spreadsheet of banking information and one of the goals is to find out the failure rate of an advertising campaign. I think I need to get a count of the total number of entries vs. the total ...
1
vote
1answer
73 views

How to visualize trending over geographical area?

I would like to visualize how a percentage evolves in time over a geographical area. In particular I have the percentage of products delivered in different regions and I want to show the weekly trend ...
2
votes
1answer
110 views

SQL Server and skip first n rows

I work for a fire and EMS dispatch agency in Florida. All of our apparatus have gps to record the longitude, latitude, timestamp, apparatus, speed, unit status and the incident number. The klaxon in ...
1
vote
2answers
960 views

Does SQL Server support the Poisson distribution?

I am trying to do some work that is primarily based in SQL Server. I cannot seem to find a native function to SQL that supports the Poisson distribution. Has anyone had success with applying Poisson ...
1
vote
0answers
17 views

Run queries against results of previously cached queries [closed]

I'm trying to find a tool or workflow that would allow a team to analysis data in multiple steps, where intermediate results are cached. (I'll assume that the raw data is in a relational database and ...