Python Tile Typecasting while Reading Data

AnwarBham
AnwarBham Contributor
in Magic ETL

A quick question in the magic ETL python Tile API method read_dataframe

can you include the dtype parameter and use a datatype dictionary

eg

import pandas as pd

import gc

dtype_dic = {'%Account_Plan_Key':'category'}

# Import the domomagic package into the script 

from domomagic import *

pd.options.mode.chained_assignment = None 

# read data from inputs into a data frame

raw = read_dataframe('Source 2', dtype=dtype_dic )

i ask because domo sets the data type of string as Object in the example im using 10k rows

if i cast this as a new column to Category it reduces the memory used considerably

but i dont want to create a subset of the columns

00:00:22.848 object
00:00:22.848 556044
00:00:22.848 Change type
00:00:22.848 category
00:00:22.848 12718


Tagged:

Comments

  • GrantSmith
    GrantSmith Coach

    Have you tried converting your series directly to a category after you import it?

    raw['column'] = raw['column'].astype('category')
    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**

Categories