Tengo una dataframe con preguntas numeradas (QID
), pero esos números ya no significaban nada, otro con los números correctos. Me gustaría actualizar el ultima con los numeros correctos. Y una vez encontrada esta coincidencia, añadimos todas las columnas que faltan de df_original a df.
df
es df_original
con una differencia: cambio unas pocas líneas que en realidad fue títulos de QuestionGrid, con nuevas líneas que también representan las respuestas de QuestionGrid. Pueden ver como lo hice en esta pregunta. Así que debería haber más lineas en df
que en df_original
.
Aqui es un estracto de la dataframe original:
>>> df_original.iloc[18:25,:10]
QID URL Questions Answers Section QType Theme Topics Answer0 Answer1
18 17 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... What is you preference of room/suites with bal... ['Preferred', 'Not preferred'] Consumer Intentions Multiple Choice Hotel ABC ['prefer', 'room', 'suit', 'balconi'] Preferred Not preferred
19 18 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... How do you want your guestroom to look like? ['Contemporary style guestrooms', 'Traditional... Consumer Intentions Multiple Choice Hotel ABC ['want', 'guestroom', 'look', 'like'] Contemporary style guestrooms NaN
20 19 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... How do you want to know about our recent offer... ['Personalized emails', 'Text messages', 'Web ... Media Consumption Multiple Choice Hotel ABC ['want', 'know', 'recent', 'offer'] Personalized emails Text messages
21 20 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... What are new offers you are most interested in? ['Not Interested', 'Somewhat Interested', 'Int... Media Consumption Multiple Choice Hotel ABC ['offer', 'interest'] Not Interested Somewhat Interested
22 21 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... How often do you want to hear on these offerings? ['Weekly', 'Bi-Weekly', 'Monthly', 'Quarterly'] Media Consumption Multiple Choice Hotel ABC ['want', 'hear', 'offer'] Weekly Bi-Weekly
23 22 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... What medium do you prefer for any change / ca... ['Through calls', 'Through text messages', 'Th... Media Consumption Multiple Choice Hotel ABC ['medium', 'prefer', 'chang', 'cancel', 'book'] Through calls Through text messages
24 23 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... Gender ['Male', 'Female', "Others (Don't Wish to Spec... Name (Optional) Multiple Choice Hotel ABC ['gender'] Male Female
Y aqui es una parte de df
:
>>>df.iloc[23:30,:10]
QID Questions QType Answer0 Answer1 Answer2 Answer3 Answer4 Answer5
23 22 How do you want to know about our recent offer... Multiple Choice Personalized emails Text messages Web Blogs Paper advertisements Video advertisements Advertisements on social media
24 23 What are new offers you are most interested in? Question Grid Not Interested Somewhat Interested Interested Highly Interested Very Highly Interested NaN
25 23_1 Discount on hotel's room charges Question Grid Not Interested Somewhat Interested Interested Highly Interested Very Highly Interested NaN
26 23_2 Hotel's reward points (Loyalty program) Question Grid Not Interested Somewhat Interested Interested Highly Interested Very Highly Interested NaN
27 23_3 Hotel's hygiene policy Question Grid Not Interested Somewhat Interested Interested Highly Interested Very Highly Interested NaN
28 23_4 Hotel medical services Question Grid Not Interested Somewhat Interested Interested Highly Interested Very Highly Interested NaN
29 23_5 In-hotel services offerings Question Grid Not Interested Somewhat Interested Interested Highly Interested Very Highly Interested NaN
Y me gustaria:
>>>df.iloc[23:30,:10]
QID Questions QType Answer0 Answer1 Answer2 Answer3 Answer4 Answer5
24 20 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... What are new offers you are most interested in? ['Not Interested', 'Somewhat Interested', 'Int... Media Consumption Question Grid Not Interested Somewhat Interested Interested Highly Interested Very Highly Interested NaN
25 20_1 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... Discount on hotel's room charges ['Not Interested', 'Somewhat Interested', 'Int... Media Consumption Discount on hotel's room charges Question Grid Not Interested Somewhat Interested Interested Highly Interested Very Highly Interested NaN
26 20_2 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... Hotel's reward points (Loyalty program) ['Not Interested', 'Somewhat Interested', 'Int... Media Consumption Hotel's reward points (Loyalty program) Question Grid Not Interested Somewhat Interested Interested Highly Interested Very Highly Interested NaN
27 20_3 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... Hotel's hygiene policy ['Not Interested', 'Somewhat Interested', 'Int... Media Consumption Hotel's hygiene policy Question Grid Not Interested Somewhat Interested Interested Highly Interested Very Highly Interested NaN
28 23_4 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... Hotel medical services ['Not Interested', 'Somewhat Interested', 'Int... Media Consumption Hotel medical services Question Grid Not Interested Somewhat Interested Interested Highly Interested Very Highly Interested NaN
29 23_5 https://docs.google.com/forms/d/1MYSjxAMCXMXB0... In-hotel services offerings ['Not Interested', 'Somewhat Interested', 'Int... Media Consumption In-hotel services offerings Question Grid Not Interested Somewhat Interested Interested Highly Interested Very Highly Interested NaN
Hasta hoy intenté:
def verify_qid(row, df_original):
for i, row_original in df_original.iterrows():
if row_original['Questions'] == row['Questions']:
row['QID'] = row_original['QID']
print(type(row_original), row_original)
for key in row_original.keys() not in row.keys():
# si la columna ya no existe en row la puestamos
row[key] = row_original[key]
for i, row in df.iterrows():
qid = row['QID']
# Verificamos qid con el de df_original
verify_qid(row, df_original) # si la pregunta es la misma pero el QID no lo es
Pero me devuelve:
<class 'pandas.core.series.Series'> QID 0
URL https://docs.google.com/forms/d/1MYSjxAMCXMXB0...
Questions When do you think your next vacation can start?
Answers ['In next 3 months', 'In next 6 months', 'In n...
Section NaN
...
11e NaN
Sum 2
Comment google forum seems broken at points needs to b...
Beyond repair NaN
Not scrapped well NaN
Name: 0, Length: 333, dtype: object
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-72-f7d6a63c90da> in <module>
13 qid = row['QID']
14 # Verificamos qid con el de df_original
---> 15 verify_qid(row, df_original) # si la pregunta es la misma pero el QID no lo es
16
17
<ipython-input-72-f7d6a63c90da> in verify_qid(row, df_original)
4 row['QID'] = row_original['QID']
5 print(type(row_original), row_original)
----> 6 for key in row_original.keys() not in row.keys():
7 # si la columna ya no existe en row la puestamos
8 row[key] = row_original[key]
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexes\base.py in __contains__(self, key)
3898 @Appender(_index_shared_docs["contains"] % _index_doc_kwargs)
3899 def __contains__(self, key) -> bool:
-> 3900 hash(key)
3901 try:
3902 return key in self._engine
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexes\base.py in __hash__(self)
3905
3906 def __hash__(self):
-> 3907 raise TypeError(f"unhashable type: {repr(type(self).__name__)}")
3908
3909 def __setitem__(self, key, value):
TypeError: unhashable type: 'Index'
Tambien intenté:
for i, row_original in df_original.iterrows():
for i, row in df.iterrows():
if row_original['Questions'] == row['Questions']:
qid = str(row['QID']).split("_")
# get the sub dataframe where the first part of the QID is the same with qid
sub_df = df.loc[df['QID'].split("_")[0] == qid[0]] # df['QID'].split("_")[0], to get the qgrid rows
# for each question in sub_df adapt their qid with row_original qid respecting "_" and provide the row values
for sub_row in sub_df.iterrows():
sub_qid = str(sub_row['QID']).split("_")
new_qid = str(row['QID'])+"_"+sub_qid[1]
sub_row['QID'] = new_qid
for key in row_original.keys() not in sub_row.keys():
# si la columna ya no existe en row la puestamos
sub_row[key] = row_original[key]
Pero me devuelve:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-77-da20e669a107> in <module>
4 qid = str(row['QID']).split("_")
5 # get the sub dataframe where the first part of the QID is the same with qid
----> 6 sub_df = df.loc[df['QID'].split("_")[0] == qid[0]] # df['QID'].split("_")[0], to get the qgrid rows
7 # for each question in sub_df adapt their qid with row_original qid respecting "_" and provide the row values
8 for sub_row in sub_df.iterrows():
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\generic.py in __getattr__(self, name)
5272 if self._info_axis._can_hold_identifiers_and_holds_name(name):
5273 return self[name]
-> 5274 return object.__getattribute__(self, name)
5275
5276 def __setattr__(self, name: str, value) -> None:
AttributeError: 'Series' object has no attribute 'split'
Actualizacion
Tambien intenté la respuesta de ansev:
#Seleccionamos la columnas de df_original a unir
#tomamos todas las que no estan en df y adicionalmente Questions y QID
cols_original_merge = ['Questions', 'QID'] + [col for col in df_original
if col not in df.columns]
#creamos una copia de df_orginal seleccionando solo las columnas
#de cols_original_merge.
#Hacemos la fusión en función de Questions.
df = df.merge(df_original[cols_original_merge], on = 'Questions', how='left')
# Se crean dos columnas QID: QID_X y QID_Y
# QID_x contiene los valores anteriores de QID
# QID_y contiene los valores nuevos de QID
# Escribimos los valores faltantes de QID_y con los valores de QID_x
# Eliminamos QID_x
# Renombramos QID_y a QID
df = (df.assign(QID_y = df['QID_y'].fillna(df['QID_x']))
.drop('QID_x', axis=1)
.rename({'QID_y' : 'QID'}, axis = 1)
)
Esta es praticamente perfecto pero ciertos numeros aparecen varias veces y otras como los alfanumericos:
>>>df.head()
QID Questions QType Answer0 Answer1 Answer2 Answer3 Answer4 Answer5 Answer6 ... 10f 11a 11b 11c 11d 11e Sum Comment Beyond repair Not scrapped well
0 0 How do you feel about your next vacation after... Checkboxes 1 2 3 4 5 6 7 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
1 0 When do you think your next vacation can start? Multiple Choice In next 3 months In next 6 months In next 1 year Only once COVID-19 is under control Only once COVID-19 vaccine is developed NaN NaN ... NaN NaN NaN 1.0 NaN NaN 2.0 google forum seems broken at points needs to b... NaN NaN
2 1 What are your preferences regarding medical tr... Likert Scale Doctor's availability in hotel Ventilator availability in hotel Tie-ups with nearby hospitals Availability of medical rooms with primary fir... NaN NaN NaN ... NaN NaN NaN NaN NaN NaN 7.0 NaN 1.0 NaN
3 2 What is your preferences of complementary brea... Multiple Choice Buffet breakfast with social distancing Buffet breakfast replaced with Ala-carte with ... Breakfast to be delivered in room with limited... Packaged breakfast only NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
4 2 What is your preference for a in-hotel grocery... Multiple Choice 1 2 3 4 5 6 7 ... NaN NaN NaN NaN NaN NaN 3.0 This could be replaced with a binary question NaN NaN
df
original elQUID=23
la pregunta esGender
, pero no es el únicoGender
de estedf
, hay 3 o 4 más que imagino son de otras preguntas, ¿Como se supone que se pueda relacionar por medio de la pregunta si está se repite?