Questions tagged [data]

Questions mostly concerned with managing data, without focus on pre-processing or modelling.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

What algorithm can I use to get the product that would give me the most profit given its buying cost, selling cost, sell volume and buy volume?

I'm working on an app for evaluating the products sold on a market in a videogame. The products can be bought for cheap and sold for expensive. I want to evaluate the best product to buy and sell, but ...
EustakiosPower's user avatar
0 votes
0 answers
20 views

How to build this data pipeline?

I don't have much experience in data engineering, so I'm here to ask for advice. I am working on a project which consists of building a dashboard for the IT department of a bank. the dashboard should ...
Wissem Boujlida's user avatar
0 votes
0 answers
15 views

Ways to check whether your predictors are even correlated with their label?

I scraped a dataset of pre-match data in a video game and am trying to classify them by outcome, i.e. wins v. losses. The models I've tried so far have poor accuracy of around 50%, so I'm thinking ...
Lilian Shi's user avatar
0 votes
1 answer
16 views

Colab file errors

and thank you to whoever answers this. I'm pretty new to data science and machine learning, and I've been trying to immerse myself in it. I feel like there's so much to learn. One of the things that's ...
Ahmed's user avatar
  • 1
1 vote
1 answer
30 views

pos_label=1 is not a valid label. Should be one of [2,4]

I am trying to retrieve my precision score but I am getting an error as follows: pos_label=1 is not a valid label. It should be one of [2 ,4] And here is the code ...
Hanh's user avatar
  • 13
0 votes
0 answers
5 views

Connect and publish with Tableau and Azure Analysis Services

My organization is migrating to another platform and we need to replicate the reports we have made with Power BI in Tableau, but the problem is that our server data is in Azure Analysis Services, but ...
Ozdran Yankovic's user avatar
0 votes
0 answers
9 views

Excluding files in Azure Data Studio when they don't have a file extension

I need to SFTP files generated every 15 minutes up to an external SAAS product. The problem is the app generating these files has a hard lock on them. It monitors realtime and keeps appending. The ...
Transitioned's user avatar
0 votes
1 answer
23 views

What are these 'code' columns in this csv and what does Pandas pivot method do with it?

In a csv discussed in the book HoML (hands on machine learning), what do the values in capital letters represent? Are these some sort of code? Also, it seems the pandas.pivot() methods does something ...
Yorick Dantuma's user avatar
0 votes
0 answers
15 views

Inverse Laplacian of xarray

my question is related to calculating the inverse Laplacian of a xarray. Specifically, I want to solve the equation ∇^2 s = h+r+e for s in Python (where ∇^2 denotes the laplacian). Here, s is an ...
random human's user avatar
0 votes
1 answer
26 views

Is it possible to use the SODA tool as open source?

I was reading about the SODA tool and was wondering if it was free to use. As far as I understand, it has a SODA library and a SODA core, but they are all just clients for connecting to a cloud server ...
JustUser's user avatar
0 votes
1 answer
21 views

How should I plot 2D data as 3D plot?

I got data (1503, 38697) (cell(datapoint), genes(features)), each elements has integer values (gene expression). so something like ...
containletters's user avatar
0 votes
0 answers
44 views

Looking for datasets on automobile parts information for machine learning

I'm embarking on a machine learning project that requires a comprehensive dataset of automobile parts information. The goal is to train a model that can identify and categorize various auto parts, ...
Anand 's user avatar
0 votes
2 answers
420 views

Solve a non-linear system, in Python, with the GAUSS-NEWTON algorithm? (Jacobian matrix J, etc.)

I would like to solve a non-linear system (which contains the goals of a football team in previous matches) using the Gauss-Netwon algorithm, in order to find the parameter (of frequency) to use as ...
Zollikofen4's user avatar
0 votes
1 answer
19 views

Hassle-free platform for a small data science team?

our team of 4 data scientists has been exploring options to automate our ETL, data storage and model updates and are looking for a hassle-free platform that can help with this. We've had several ...
Rotten_Cabbagges's user avatar
0 votes
0 answers
12 views

Packet data needed fot the model

I want to use this data set https://huggingface.co/datasets/rdpahalavan/network-packet-flow-header-payload for network attack classification Here is packet structure from the README file. Assuming I ...
Roma's user avatar
  • 101
0 votes
0 answers
23 views

statistical analysis of two data sets have different number of sampling

I have two datasets from my statistical fault analysis: one with 400 iterations and another with 500 iterations. Both datasets have the same maximum and minimum values, and some data points are ...
Fadi's user avatar
  • 1
1 vote
1 answer
56 views

Should I apply the same data transformations in production for my classification model's inference steps

I am now moving my best classification model to production and doing tests currently. Should I use the same scaler() I used in training during my inference in ...
Marc Atanante's user avatar
2 votes
2 answers
82 views

How to deal with high data volumes? (Tools, techniques, concepts, etc.)

I have some doubts about how to deal with high volumes of data. I'm currently working in the data analysis/data science field, so I've had the chance to perform calculations, manipulate data, and ...
tms's user avatar
  • 21
0 votes
0 answers
5 views

Data Tracking For Process Improvement

I work in a manufacturing plant with 6 production lines. As part of the maintenance crew, it is my job to record all maintenance work performed on the equipment inside of our shared OneNote. This ...
Austin's user avatar
  • 1
0 votes
0 answers
24 views

What was used before data augmentation (e.g. SMOTE) to train ML models on imbalanced data? Please provide citations

I am seriously curious on how imbalance data was treated in machine learning and statistical learning before modern time data augmentation solutions such as SMOTE appeared. Please provide citations ...
Full Array's user avatar
0 votes
1 answer
65 views

Loading Data with column Headers

I have a file called adult.data that contains my test set in a CSV file format. Unfortunately there are no column headers in the file to describe the variables. But I have a file called adult.names ...
Sean's user avatar
  • 1
0 votes
0 answers
50 views

Outliers problem

everyone hope you all are good. I have a hard time with outliers First i found some outliers in train set then i capped them and did the same with test set. So when i checked my rsme root mean ...
Shoaib Ahmed's user avatar
0 votes
0 answers
12 views

Seeking Guidance on Creating Sample Data for AI Business Use Case

Subject: Seeking Guidance on Creating Sample Data for AI Business Use Case Dear StackExhange Family I am currently working on an AI project aimed at providing AI embaded solutions to HealthCare, ...
TARUN KUMAR's user avatar
1 vote
0 answers
29 views

Determining the start and stop of vehicle movement by GPS

There is a large fleet, slightly more than a million vehicles, which is constantly growing. Each vehicle sends GPS coordinates to the server, as well as events (ignition is on, door is open, parking ...
Abraham Lincoln's user avatar
0 votes
0 answers
28 views

Data Science Communities for complete beginners

Any recommendation as to where i can join communities and forums so i can get to learn Data Science please? i am a total beginner and i'm having a hard time learning it alone.
Jibril Yahaya Jibril's user avatar
0 votes
1 answer
30 views

Very basic python question [closed]

I am doing the free interactive learn python tutorial, and I am on the first conditions lesson. The task was to make it so that every statement in the following code resolved true. The code below was ...
auditorchick's user avatar
0 votes
0 answers
13 views

Pre-trained text classifier to check whether a document is business or personal

I need a pre-trained classifier that checks if a document is either a business or personal document. I don't have time to train one myself so if anyone knows of a classifier that is pre-trained and ...
user155097's user avatar
0 votes
0 answers
11 views

in data science if a feature has a dominant category we drop it before model building. why do we do so?

In machine learning and data science, we drop the feature in which one category heavily dominates the other category. Why do we do so?
Lakshya Kumar's user avatar
0 votes
1 answer
17 views

Dates countries joined/left international organisations

Does anyone know of a reliable data source containing the dates countries left or joined international organisations, for example,WTO, EU. etc? Ideally I'd like this going back to 1946. Even the year ...
James's user avatar
  • 121
0 votes
0 answers
9 views

Which Frameworks/Libs Best Support Integer-Based Features, Scaling, Training, etc?

Papers such as Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference have interested me in exploring integer-based data science. In particular, I'm thinking of ...
ezekiel68's user avatar
0 votes
0 answers
26 views

Using PowerBI calculated columns vs calculated measures to calculate a frequency flag column

Sorry for the long question. In PowerBI I have fields called 'user ID' and 'service'. Each row represents an activity, so may not be unique. I want to create a new variable called '5+ flag' that for ...
Molly's user avatar
  • 1
0 votes
0 answers
7 views

Terminology: What is it called when the filter criteria is added to the data table?

When we want to focus on a group within our dataset, it is common to filter it. Suppose we have a data table showing how many points 4 players earned in a game. Table 1: Game Scores for 4 Players Age ...
madprogramer's user avatar
0 votes
2 answers
37 views

Lack of imagination when creating projects

Whenever I find a set of data to carry out a project, I end up running out of ideas and don't know how to proceed with the project. I don't know how to proceed or what information to look for. How do ...
Davi Komura de Castro's user avatar
0 votes
0 answers
15 views

Mapping two unrelated representation spaces with different distributions, roughly preserving similarities

I've got two embedding / representation spaces which are completely unrelated to begin with, yet I wish to find a (=any) mapping between them. Space A are feature histograms with a somewhat normal ...
nussbaum's user avatar
0 votes
0 answers
11 views

How should I save data in deeplearning with nosql (mongodb)?

I usually use file system to manage data for my deep learning model, but one of my boss told me to make nosql database to manage data. Datasets I use have m rows, and n columns of count matrix and ...
containletters's user avatar
0 votes
0 answers
12 views

Detect Recent Anomalies

I'm trying to see how to approach this problem: I have a dataset of fraud transactions. There are several categorical columns, like country, type, merchant, etc. All of the records are considered ...
F_M's user avatar
  • 131
0 votes
0 answers
12 views

(Apparently) Inconsistent Results Between Visual Patterns and Chi-Square Test in Categorical Data Analysis

I'm stuck while working on a categorical data analysis project and would appreciate any insight. Here are the first 4 rows of the dataset: successful_upload feat_1 feat_2 0 T>2 <\$10K 1 0.5&...
Maby Esperanza's user avatar
0 votes
0 answers
15 views

Data processing for Events that repeat a fixed number of times in each group in machine learning

Assume I have a football teams historical match performance data at player level, with attributes like number of goals score by each player, other performance metrices like passes given, crosses done, ...
imhaka's user avatar
  • 1
0 votes
0 answers
73 views

I have a data set that having profit column with negative values. what should I do?

I am working on a data set that data set have a profit column with negative values what should I do can we remove it or ignore it .
akshay gaikwad's user avatar
0 votes
1 answer
194 views

How would normalizing be affected by outliers? And how to avoid it?

I have a data set that boils down to Three clomuns: 1.Supplier name 2. Number of transactions with supplier 3. Total value of those transaction. I'm trying to find the best way to rank all suppliers ...
Rakuzan's user avatar
0 votes
0 answers
16 views

Are there any publications that deal with decreasing value of data as time decreases toward an event?

I'm looking for research related to the value of data volume as time decreases approaching an event. For example, large data sets are needed to understand changes in geological and meteorological ...
Matt E.'s user avatar
0 votes
0 answers
43 views

What is the best way to approach anomaly detection on a data set using machine learning?

I am looking to help on where to start exploring machine learning when it comes to data processing. Say I have the following csv file with hundreds of thousands of rows of data: ID Amount Overdue (...
Doug's user avatar
  • 1
0 votes
0 answers
21 views

Statistical approach for comparing ranks

I'm looking for a statistical approach to compare ranks produced by 2 versions of an ranking algorithm (A & B) against the actual ranks in the system. This is about ranking hospitals and the ...
museshad's user avatar
  • 101
0 votes
1 answer
53 views

Converting categorical to the percentage

How do I convert the categorical value to the percentage?| I have this asset wellness data: Poor: 3 Warning: 27 Good: 120 How do I convert it to the percentage ...
Muhammad Ikhwan Perwira's user avatar
0 votes
2 answers
291 views

Open retail/ecommerce dataset with customers, transactions, products, loyalty, discounts, promotion information?

Does anyone know of an open retail/e-commerce dataset with information about customers, transactions, products, loyalty, discounts, and promotion information? I am planning to create 360-degree ...
exan's user avatar
  • 101
0 votes
0 answers
10 views

Data Validity Update

My question is simple. I want to run Orange to predict where will road accidents take place on a highway. So, I will: Upload the dataset from excel with a few columns and rows of information to be ...
Etzel's user avatar
  • 1
1 vote
2 answers
43 views

Tips for scraping crypto data in the right way

I am scraping data from crypto site and want to use neural network algorithm for predicting data. the way i save data is like these: and there is bunch of other features like open/high/low/close for ...
mohammad ariyan rad's user avatar
0 votes
0 answers
20 views

Stacking realization problems

I have two dataframes: x_train with features got from base models and y_train with ground true labels of these features using cross_validation. ...
XEX's user avatar
  • 3
0 votes
0 answers
15 views

Multiple coords to one data_vars in xr.dataset

I am trying to create a xarray dataset for a list of events (labeled 1 or 0, so it's an array of integers). To each event three numbers need to be attached - a date, longitude, and latitude. I provide ...
Shaz's user avatar
  • 125
0 votes
0 answers
17 views

Gaussian 'around' a given distribution

I want to find the data points within 0.2 dex of the Main Sequence Relation: (The main sequence relation is what you can observe in this figure) Dex is an astronomical jargon which signifies scatter, ...
Ambica Govind's user avatar

1
2 3 4 5
18