1

Tengo un archivo de datos (300MB) con columnas desgastadas y un archivo de nombre de columna. Me gustaría transformar el archivo de datos cambiando los nombres de las columnas. Por el momento el archivo de datos es:

    Unnamed: 0  id  member_id   loan_amnt   funded_amnt funded_amnt_inv term    int_rate    installment grade   sub_grade   emp_title   emp_length  home_ownership  annual_inc  verification_status issue_d loan_status pymnt_plan  url desc    purpose title   zip_code    addr_state  dti delinq_2yrs earliest_cr_line    inq_last_6mths  mths_since_last_delinq  mths_since_last_record  open_acc    pub_rec revol_bal   revol_util  total_acc   initial_list_status out_prncp   out_prncp_inv   total_pymnt total_pymnt_inv total_rec_prncp total_rec_int   total_rec_late_fee  recoveries  collection_recovery_fee last_pymnt_d    last_pymnt_amnt next_pymnt_d    last_credit_pull_d  collections_12_mths_ex_med  mths_since_last_major_derog policy_code application_type    annual_inc_joint    dti_joint   verification_status_joint   acc_now_delinq  tot_coll_amt    tot_cur_bal open_acc_6m open_act_il open_il_12m open_il_24m mths_since_rcnt_il  total_bal_il    il_util open_rv_12m open_rv_24m max_bal_bc  all_util    total_rev_hi_lim    inq_fi  total_cu_tl inq_last_12m    acc_open_past_24mths    avg_cur_bal bc_open_to_buy  bc_util chargeoff_within_12_mths    delinq_amnt mo_sin_old_il_acct  mo_sin_old_rev_tl_op    mo_sin_rcnt_rev_tl_op   mo_sin_rcnt_tl  mort_acc    mths_since_recent_bc    mths_since_recent_bc_dlq    mths_since_recent_inq   mths_since_recent_revol_delinq  num_accts_ever_120_pd   num_actv_bc_tl  num_actv_rev_tl num_bc_sats num_bc_tl   num_il_tl   num_op_rev_tl   num_rev_accts   num_rev_tl_bal_gt_0 num_sats    num_tl_120dpd_2m    num_tl_30dpd    num_tl_90g_dpd_24m  num_tl_op_past_12m  pct_tl_nvr_dlq  percent_bc_gt_75    pub_rec_bankruptcies    tax_liens   tot_hi_cred_lim total_bal_ex_mort   total_bc_limit  total_il_high_credit_limit  revol_bal_joint sec_app_earliest_cr_line    sec_app_inq_last_6mths  sec_app_mort_acc    sec_app_open_acc    sec_app_revol_util  sec_app_open_act_il sec_app_num_rev_accts   sec_app_chargeoff_within_12_mths    sec_app_collections_12_mths_ex_med  sec_app_mths_since_last_major_derog hardship_flag   hardship_type   hardship_reason hardship_status deferral_term   hardship_amount hardship_start_date hardship_end_date   payment_plan_start_date hardship_length hardship_dpd    hardship_loan_status    orig_projected_additional_accrued_interest  hardship_payoff_balance_amount  hardship_last_payment_amount    disbursement_method debt_settlement_flag    debt_settlement_flag_date   settlement_status   settlement_date settlement_amount   settlement_percentage   settlement_term
0   1040017 NaN NaN 14000   14000   14000.0 36 months   12.69   469.63  C   C2  Receiving Dock Worker   9 years MORTGAGE    40000.0 Not Verified    2015-10-01  Charged Off n   NaN NaN debt_consolidation  Debt consolidation  166xx   PA  17.07   0.0 Jun-2001    1.0 NaN NaN 5.0 0.0 5848    90.0    15.0    f   0.0 0.0 6057.790000 6057.79 4091.51 1556.41 0.0 409.87  73.7766 Oct-2016    469.63  NaN Jul-2018    0.0 NaN 1   Individual  NaN NaN NaN 0.0 0.0 119776.0    NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 6500.0  NaN NaN NaN 4.0 23955.0 2167.0  90.0    0.0 0.0 141.0   172.0   3.0 3.0 1.0 3.0 NaN 3.0 NaN 0.0 3.0 3.0 8.0 8.0 6.0 3.0 8.0 3.0 5.0 NaN 0.0 0.0 2.0 100.0   100.0   0.0 0.0 123292.0    29809.0 6500.0  25992.0 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN N   NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN Cash    N   NaN NaN NaN NaN NaN NaN
1   1050463 NaN NaN 1000    1000    1000.0  36 months   9.17    31.88   B   B2  Portfolio Manager   1 year  MORTGAGE    80000.0 Verified    2015-10-01  Fully Paid  n   NaN NaN credit_card Credit card refinancing 949xx   CA  12.51   0.0 Oct-1967    3.0 NaN 22.0    9.0 1.0 7634    37.2    32.0    w   0.0 0.0 1021.730000 1021.73 999.99  21.74   0.0 0.00    0.0000  Feb-2016    27.85   NaN Feb-2017    0.0 NaN 1   Individual  NaN NaN NaN 0.0 0.0 53994.0 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 20500.0 NaN NaN NaN 4.0 5999.0  12866.0 37.2    0.0 0.0 188.0   575.0   4.0 4.0 3.0 4.0 NaN 1.0 NaN 0.0 3.0 3.0 6.0 16.0    9.0 6.0 20.0    3.0 9.0 0.0 0.0 0.0 3.0 100.0   0.0 1.0 0.0 80788.0 53994.0 20500.0 60288.0 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN N   NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN Cash    N   NaN NaN NaN NaN NaN NaN
...

El archivo con los nombres tiene el siguiente aspetos:

introducir la descripción de la imagen aquí

Por ejemplo me gustaria que issue_d en el header de la dataframe se transforma en the month which the loan was funded.

1
  • Podrías explicar un poco más qué cómo son exactamente los cambios que quieres? he entendido que que quieres poner los nombres que hay en el archivo de nombres en los encabezados de las columnas del csv, pero todos los que he mirado aparecen igual, así que no me queda muy claro qué es lo que necesitas. Commented el 10 sept. 2019 a las 16:32

1 respuesta 1

1

No me parece muy buena idea poner nombres de columnas tan largos (uno de ellos sería "The listed amount of the loan applied for by the borrower. If at some point in time, the credit department reduces the loan amount, then it will be reflected in this value."), pues esos son los nombres que luego tendrías que usar entre corchetes para seleccionar la correspondiente columna...

En todo caso, la idea es:

  • Carga tu csv con las "traducciones" de nombre corto a nombre largo, por ejemplo usando pandas para que lo cargue en un dataframe
  • Convierte eso que has leido en un diccionario en el que las claves serían los nombres cortos y los valores los nombres largos. Pandas puede hacerlo también.
  • Usa sobre el dataframe que quieres renombrar, llamémosle df el método rename() pasándole ese diccionario y el valor axis=1.

Es decir:

df = pd.read_csv("Tu-archivo-de-datos.csv")
df2 = pd.read_csv("data_dictionary.csv", header=None, encoding="latin1").set_index(0)
traduccion = df2[1].to_dict()

df = df.rename(traduccion, axis=1)

Tu nuevo df tendrá ahora esta pinta:

>>> df.head()
   Unnamed: 0  ...  The number of months that the borrower will be on the settlement plan
0     1040017  ...                                                NaN                    
1     1050463  ...                                                NaN                    
2     1056254  ...                                                NaN                    
3     1013860  ...                                                NaN                    
4     1018431  ...                                                NaN                    

[5 rows x 146 columns]

>>> df.columns
Index(['Unnamed: 0', 'A unique LC assigned ID for the loan listing.',
       'A unique LC assigned Id for the borrower member.',
       'The listed amount of the loan applied for by the borrower. If at some point in time, the credit department reduces the loan amount, then it will be reflected in this value.',
       'The total amount committed to that loan at that point in time.',
       'The total amount committed by investors for that loan at that point in time.',
       'The number of payments on the loan. Values are in months and can be either 36 or 60.',
       'Interest Rate on the loan',
       'The monthly payment owed by the borrower if the loan originates.',
       'LC assigned loan grade',
       ...
       'The payoff balance amount as of the hardship plan start date',
       'The last payment amount as of the hardship plan start date',
       'The method by which the borrower receives their loan. Possible values are: CASH, DIRECT_PAY',
       'Flags whether or not the borrower, who has charged-off, is working with a debt-settlement company.',
       'The most recent date that the Debt_Settlement_Flag has been set  ',
       'The status of the borrowers settlement plan. Possible values are: COMPLETE, ACTIVE, BROKEN, CANCELLED, DENIED, DRAFT',
       'The date that the borrower agrees to the settlement plan',
       'The loan amount that the borrower has agreed to settle for',
       'The settlement amount as a percentage of the payoff balance amount on the loan',
       'The number of months that the borrower will be on the settlement plan'],
      dtype='object', length=146)

Aunque no sé si será esto lo que querías hacer o te he entendido mal, pues sigo pensando que no es buena idea unos nombres de columna tan kilométricos (ya ves que al ser tan largos pandas apenas puede mostrar uno solo, con sus opciones de display por defecto)

Tu Respuesta

By clicking “Publica tu respuesta”, you agree to our terms of service and acknowledge you have read our privacy policy.

¿No es la respuesta que buscas? Examina otras preguntas con la etiqueta o formula tu propia pregunta.