o ҷh @sddlmZddlmZddlmZddlmZddlm Z ddl m Z ddl Z ddl mZdd lmZmZdd lmZerDdd lmZGd d d eZdS)) annotations) TYPE_CHECKING)using_pyarrow_string_dtype)lib)import_optional_dependency) is_integerN) DataFrame)_arrow_dtype_mappingarrow_string_types_mapper) ParserBase) ReadBuffercsHeZdZdZdfdd Zdd Zdd d ZdddZdddZZ S)ArrowParserWrapperz7 Wrapper for the pyarrow engine for read_csv() srcReadBuffer[bytes]returnNonec s$t|||_||_|dS)N)super__init__kwdsr _parse_kwds)selfrr __class__Y/var/www/html/venv/lib/python3.10/site-packages/pandas/io/parsers/arrow_parser_wrapper.pyrs  zArrowParserWrapper.__init__cCsN|jd}|dur dn||_|jd}t|trtdt|jd|_dS)z? Validates keywords before passing to pyarrow. encodingNzutf-8 na_valuesz?The pyarrow engine doesn't support passing a dict for na_values)rgetr isinstancedict ValueErrorlistr)rrrrrrr%s   zArrowParserWrapper._parse_kwdscCsdddddd}|D]\}}||jvr&|j|dur&|j||j|<q |j}t|tr3|g}nd}||jd<d d |jD|_d d |jD|_d |jdv|jd <|j du|j durf|j n|jd|j d|_ dS)z: Rename some arguments to pass to pyarrow include_columns null_values escape_charignore_empty_lines decimal_point)usecolsr escapecharskip_blank_linesdecimalNtimestamp_parserscS&i|]\}}|dur|dvr||qS)N) delimiter quote_charr$r%r.0 option_name option_valuerrr Qz;ArrowParserWrapper._get_pyarrow_options..cSr,)N)r"r# true_values false_valuesr&r+rr/rrrr3Xr4strings_can_be_nullskiprows)autogenerate_column_names skip_rowsr) itemsrrpop date_formatrstr parse_optionsconvert_optionsheaderr read_options)rmapping pandas_name pyarrow_namer>rrr_get_pyarrow_options3s8    z'ArrowParserWrapper._get_pyarrow_optionsframerc stj}d}|jdur7|jdur|jdurt||_t|j|kr3tt|t|j|j|_d}|j_|j\}|jdur|j}t |jD]S\}}t |r_j|||<n |jvrlt d|d|j dur|j |dur||j |fn j||j j|f\}} | dur|| |<|j |=qOj|ddd|jdur|sdgtjjj_|j durt|j trՇfdd|j D|_ z |j WSty} zt | d} ~ wwS) z Processes data read in based on kwargs. Parameters ---------- frame: DataFrame The DataFrame to process. Returns ------- DataFrame The processed DataFrame. TNFzIndex z invalid)dropinplacecs i|] \}}|jvr||qSr)columns)r0kvrHrrr3s z>ArrowParserWrapper._finalize_pandas_output..)lenrKrBnamesranger!_do_date_conversions index_colcopy enumeraterr dtyperastype set_indexindexrrr< TypeError) rrHnum_colsmulti_index_named_ index_to_setiitemkey new_dtypeerrNr_finalize_pandas_outputosR           z*ArrowParserWrapper._finalize_pandas_outputc Cs*td}td}||j|j|jdi|j|jdi|j|jdi|j d}|j d}|t j ur^|j }|}t|j jD]\}}|j|rX|||||}qB||}|dkrj|jtjd} n&|dkrt} t| |<|j| jd} ntr|jtd} n|} || S) z Reads the contents of a CSV file into a DataFrame and processes it according to the kwargs passed in the constructor. Returns ------- DataFrame The DataFrame created from the CSV file. pyarrowz pyarrow.csv)rCr@rA dtype_backend) types_mappernumpy_nullableNr) rrGread_csvr ReadOptionsrC ParseOptionsr@ConvertOptionsrArr no_defaultschemafloat64rUtypesis_nullsetfield with_typecast to_pandaspd ArrowDtyper Int64Dtypenullrrr rd) rpa pyarrow_csvtablerf new_schemanew_typer_ arrow_typerH dtype_mappingrrrreads<      zArrowParserWrapper.read)rrrr)rr)rHrrr)rr) __name__ __module__ __qualname____doc__rrrGrdr __classcell__rrrrr s  <Cr ) __future__rtypingrpandas._configr pandas._libsrpandas.compat._optionalrpandas.core.dtypes.inferencerpandasrwrpandas.io._utilr r pandas.io.parsers.base_parserr pandas._typingr r rrrrs