0
$\begingroup$

I'm using a s3 bucket to store a model I trained in python. Since I'm using an s3 bucket I convert the file to binary first and then store it on the bucket.

with open(r'model_svc_kernel_linear02.sav',"rb") as f:
    s3_client.upload_fileobj(f, bucket, "model.sav")

I'm able to read the file, however I can't use the model to predict since its only a bytes object.

model =  s3_resource.Object('mybucket', 'model.sav').get()['Body'].read()

When I try to read the binary file I get this error

with open(model) as f:
    contents = f.read()

Traceback (most recent call last):
  File "<string>", line 1, in <module>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

I've been trying different encodings but none seem to be working. Is there a better way to store the model on the s3 bucket, read it and use it to make a prediction?

$\endgroup$

1 Answer 1

0
$\begingroup$

To use the model stored in an s3 bucket for prediction, you can first load the model into memory by calling the loads() method of the pickle module, which converts the binary data into a Python object. Then, you can use the loaded model to make predictions.

Here is an example of how to load the model and use it for prediction:

import boto3
import pickle

s3_client = boto3.client('s3')

# Download the model from the s3 bucket
response = s3_client.get_object(Bucket='mybucket', Key='model.sav')

# Read the binary data from the response
binary_data = response['Body'].read()

# Load the model from the binary data
model = pickle.loads(binary_data)

# Use the model for prediction
predictions = model.predict(X)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.