Questions tagged [aws]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

How can I preview AWS Glue jobs?

I´m want to use Glue to extract data from an RDS PostgresDB, transform/clean it and load into an S3 Bucket so I can use Athena and Quicksight to visualize the data and create reports. I´m currently ...
1
vote
1answer
24 views

Does AWS Sagemaker continue running when I go offline?

I'm pulling data from web services in an AWS Sagemaker Notebook. I'd like to be able to close my computer for an hour, reconnect my computer and see that the data collection has continued to run. ...
0
votes
0answers
12 views

pd.read_csv() not reading /tmp folder AWS Lambda

I am using AWS Lambda to download a CSV from S3 and then upload each recod to a mongo. I download the csv to /tmp folder using the boto3 client and with pandas read it with pd.read_csv(). However the ...
0
votes
0answers
7 views

What's the difference between two kinds of deployment in sagemaker

I read the deployment model in stagemaker from sagemaker webistes: https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-deployment.html It says there are two ways for deployment: 1. via AWS ...
0
votes
0answers
27 views

Python data cleaning for CSV using Pandas and PySpark and mapping to JSON in AWS ETL Glue Environment

I have recently started working on some ETL work and wanted some guidance in this area related to data cleaning from CSV to JSON mapping using AWS Glue, Python (pandas, pyspark). Question Need a ...
0
votes
0answers
17 views

AWS comprehend service topic modelling takes too much time

I have an AWS Comprehand service in which i created an analysis job for topic modelling.Input to job was just 4.4kB text file, I got correct output after 25-30 minutes, then i tried with 750kB file it ...
0
votes
0answers
15 views

aws sagemaker data input configuration

I m trying to run this training job on the sagemaker but it doesn't like the data I uploaded in the s3 bucket. I simply uploaded ...
0
votes
1answer
10 views

Intermittent type error when running CNN inside Docker container

I've been working on a simple computer vision API with a few endpoints for grabbing useful information from ebay images. The API lives in a docker container that looks for the h5 files, downloads ...
1
vote
0answers
18 views

Where is $SPARK_HOME on an EC2 instance?

I am working on an EC2 instance with anaconda and pyspark pre-installed. My $SPARK_HOME variable is not set. Any ideas on how to locate my spark home?
0
votes
0answers
10 views

Hints on my EMR Spark Cluster

I am new to Spark, in particular I am tying to find out which are the best recommended properties to set to work on a 1 master 2 slaves cluster with 10GB memory each. Do you know which are a good set ...
3
votes
2answers
340 views

sklearn and pandas in AWS Lambda

I made a front end where I would like to make REST calls to an AWS Lambda interfaced with AWS API Gateway. I dumped my model as a pickle file (and so my encoders) which I initially trained locally. ...
0
votes
1answer
105 views

Difference between AWS amazonei_tensorflow and tensorflow conda environments

In AWS SageMaker there are several Conda environments available based on different ML frameworks: conda info -e I would like to know the difference between a <...
0
votes
0answers
21 views

Unable to parse manifest sagemaker

I have a csv stored in S3 and try to build a multiclass XGB training job with Sage Maker. Even though I've put the target variables (classes from 1 to 3) in the firs column and removed the headers, ...
1
vote
0answers
116 views

Should I remove the trend from timeseries when using DeepAR

I saw that for some other algorithms for timeseries data it is advised to remove trend and seasonality before doing the prediction (ex: ARIMA and LSTM) I figured out from the paper that SageMaker's ...
2
votes
2answers
532 views

Keras neural network entirely different results on two different hardwares (one is AWS)

I have a Keras model. which is defined as follows: ...
2
votes
2answers
511 views

AWS EC2 Segmentation Fault (core dumped) issue with fast-style-transfer

I'm trying to utilize AWS EC2 p2.xlarge instance to convert images using style-transfer code given in this git repo:https://github.com/lengstrom/fast-style-transfer.git, yet when the input file ...
1
vote
0answers
21 views

How to prepare future data for training

Assume I have a large data of ecommerce website sessions with user id key (a user can have multiple sessions with random time between them). The data is on S3 in json gzipped format. On some sessions ...
1
vote
1answer
51 views

How can I split an image into rectangles?

I have a labelled form to which people will add their name and a series of numbers. They will then take a picture of the form. Like so: I can get decent results by simply sending this to AWS ...
1
vote
0answers
273 views

Get deepAR fit() metrics to python

I use deepAR RNN on AWS via python. I run the estimator.fit(inputs=data_channels) and want to get insight on the algorithm performance and the fit that the model ...
0
votes
2answers
210 views

deepAR RNN from AWS Sagemaker - should I clean the data first?

I test the Sagemaker AWS solution for RNN: deepAR. Previously I used sklearn for this and obviously I cleaned the data to avoid highly correlated time series (KBest,...
1
vote
1answer
575 views

How do I get the name of Sagemaker Estimator's job

I'm having a stumbling block with SageMaker. How do I know what my job name is? For example: ...
2
votes
1answer
2k views

Why is Amazon P2 instance cost much more than G3?

I'm currently looking for the correct number of 2018 budget for the AWS instances in preparation of more deep neural net training, but now a bit confused because of this chart below: p2.16xlarge ...
1
vote
0answers
58 views

maximum dimensionality of AWS Machine Learning

I am toying with AWS Machine Learning and I have a dataset with about 200 records with about 220,000 variables each. Apparently, AWS Machine Learning has a hard limit of 1000 variables. How can I ...
0
votes
2answers
4k views

Does any cloud service provide free GPU access?

Is there any cloud developer platform which provides free access to an NVIDIA GPU instance, maybe only for a limited trial period? I'd like to work on it before committing to a paid option. I have ...
3
votes
1answer
551 views

Using AWS ML to recommend products

I have millions of user ratings on about 2k products. I want to use Machine Learning to analyse these ratings and recommend products to users based on other users ratings of the same and different ...
0
votes
1answer
21 views

AWS model performance evaluation interpretation

I've trained and testing a logistic regression binary-classification model using AWS Machine Learning. The evaluation has the ...
4
votes
2answers
560 views

AWS machine learning prediction schema problems

I've trained an AWS Machine Learning model with the training data from here : https://www.kaggle.com/c/titanic/data I'm now trying to run a batch prediction with the test data from the same source ...
1
vote
1answer
642 views

How do I access data on my EBS Volume from R-Studio Server on Ubuntu EC2 Instance

I have setup R-Studio Server on an Ubuntu EC2 instance for the first time and successfully started r-studio server in my browser. I also have putty ssh client. How do I set path in r-studio server to ...
10
votes
4answers
9k views

Downloading a large dataset on the web directly into AWS S3

Does anyone know if it's possible to import a large dataset into Amazon S3 from a URL? Basically, I want to avoid downloading a huge file and then reuploading it to S3 through the web portal. I ...
0
votes
1answer
1k views

Can hadoop with Spark be configured with 1GB RAM

I'm trying to set up a cluster (1 namenode, 1 datanode) on AWS. I'm using free one year trial period of AWS, but the challenge is, instance is created with 1GB of RAM. As I'm a student, I cannot ...
6
votes
2answers
1k views

Processing data stored in Redshift

We're currently using Redshift as our data warehouse, which we're very happy with. However, we now have a requirement to do machine learning against the data in our warehouse. Given the volume of data ...
4
votes
3answers
293 views

What technologies are fastest at performing joins on large datasets?

By "large", I mean in the range of 100m to 10b rows. I'm currently using both Hadoop MapReduce and Amazon RedShift. MapReduce has been a little disappointing here. Redshift works very well if the ...
12
votes
3answers
3k views

Does Amazon RedShift replace Hadoop for ~1XTB data?

There is plenty of hype surrounding Hadoop and its eco-system. However, in practice, where many data sets are in the terabyte range, is it not more reasonable to use Amazon RedShift for querying ...
12
votes
3answers
1k views

Instances vs. cores when using EC2

Working on what could often be called "medium data" projects, I've been able to parallelize my code (mostly for modeling and prediction in Python) on a single system across anywhere from 4 to 32 cores....