I am selecting data from Amazon Redshift Table with 500 millions rows. I have 64bit python installed.
import psycopg2 from sqlalchemy import create_engine import pandas as pd engine = create_engine('postgresql://'username':pwd@host/dbname') data_frame = pd.read_sql_query('SELECT * FROM table_name ;', engine)
Everytime I run the code I get a "Out of Memory error". I have 16gb Ram. I am not sure how to resolve this issue.
Would really appreciate any help on this! Thanks