Tengo una dataframe
QID URL Questions Answers Section QType Theme Topics Answer0 Answer1 Answer2 Answer3 Answer4 Answer5 Answer6
1113 1096 https://docs.google.com/forms/d/1hIkfKc2frAnxsQzGw_h4bIqasPwAPzzLFWqzPE3B_88/edit?usp=sharing To what extent are the following factors considerations in your choice of flight? ['Very important consideration', 'Important consideration', 'Neutral', 'Not an important consideration', 'Do not consider'] When choosing an airline to fly with, which factors are most important to you? (Please list 3.) Multiple Choice Airline XYZ ['extent', 'follow', 'factor', 'consider', 'choic', 'flight'] Very important consideration Important consideration Neutral Not an important consideration Do not consider NaN NaN
1116 1097 https://docs.google.com/forms/d/1hIkfKc2frAnxsQzGw_h4bIqasPwAPzzLFWqzPE3B_88/edit?usp=sharing How far in advance do you typically book your tickets? ['0-2 months in advance', '2-4 months in advance', '4-6 months in advance', '6-8 months in advance', '8-10 months in advance', '10-12 months in advance', '12+ months in advance'] When choosing an airline to fly with, which factors are most important to you? (Please list 3.) Multiple Choice Airline XYZ ['advanc', 'typic', 'book', 'ticket'] 0-2 months in advance 2-4 months in advance 4-6 months in advance 6-8 months in advance 8-10 months in advance 10-12 months in advance 12+ months in advance
con filas de los cuales quiero cambiar unas pocas líneas que en realidad son títulos de QuestionGrid, con nuevas líneas que también representan las respuestas. Tengo un otra, Pickle, que contiene la información para construir las líneas que actualizarán las antiguas. Cada vez una línea antigua se transformará en varias líneas nuevas (lo especifico porque no sé cómo hacerlo).
Estas lineas solo son los titulos de grid de preguntas como la siguiente:
dataframe esperado
Me gustaria insertarlos en la dataframe original, en lugar de las lineas donde coinciden en la columna de Questions
, como en el siguiente dataframe:
QID Questions QType Answer1 Answer2 Answer3 Answer4 Answer5
1096 'To what extent are the following factors considerations in your choice of flight?' Question Grid 'Very important consideration' 'Important consideration' 'Neutral' 'Not an important consideration' 'Do not consider'
1096_S01 'The airline/company you fly with'
1096_S02 'The departure airport'
1096_S03 'Duration of flight/route'
1096_S04 'Baggage policy'
1097 'To what extent are the following factors considerations in your choice of flight?' Question Grid ...
1097_S01 ...
...
El mio intento
Intenté utilisar la respuesta de gcoronel99 sobre la representacion de una grid de preguntas en un marco de datos.
import pickle
qa = pd.read_pickle(r'Python/interns.p')
df = pd.read_csv("QuestionBank.csv")
def isGrid(dic, df):
'''Check if a row is a row related to a Google Forms grid
if it is a case update this row'''
d_answers = dic['answers']
try:
answers = d_answers[2]
if len(answers) > 1:
# find the line in df and replace the line where it matches by the lines
update_lines(dic, df)
return df
except TypeError:
return df
def update_lines(dic, df):
'''find the line in df and replace the line where it matches
with the question in dic by the new lines'''
lines_to_replace = df.index[df['Questions'] == dic['question']].tolist() # might be several rows and maybe they aren't all to replace
# I check there is at least a row to replace
if lines_to_replace:
# I replace all rows where the question matches
for line_to_replace in lines_to_replace:
# replace this row and the following by the following dataframe
questions = reduce(lambda a,b: a + b,[data['answers'][2][x][3] for x in range(len(data['answers'][2]))])
ind_answers = dic["answers"][2][0][1]
answers = []
# I get all the potential answers
for i in range(len(ind_answers)):
answers.append(reduce(lambda a,b: a+b,[ind_answers[i] for x in range(len(questions))])) # duplicates as there are many lines with the same answers in a grid, maybe I should have used set
answers = {f"Answer{i}": answers[i] for i in range(0, len(answers))} # dyanmically allocate to place them in the right columns
dict_replacing = {'Questions': questions, **answers} # dictionary that will replace the forle create the new lines
df1 = pd.DataFrame(dict_replacing)
df1.index = df1.index / 10 + line_to_replace
df = df1.combine_first(df)
return df
Hicé un Colaboratory notebook si quieren utilizarlo.
Lo que obtengo
Pero la dataframe esta de la misma tamana antes y despues de hacer esto. En efecto, obtengo:
QID Questions QType Answer1 Answer2 Answer3 Answer4 Answer5
1096 'To what extent are the following factors considerations in your choice of flight?' Question Grid 'Very important consideration' 'Important consideration' 'Neutral' 'Not an important consideration' 'Do not consider'
1097 'To what extent are the following factors considerations in your choice of flight?' Question Grid ...
Actualizacion
Logré obtener las filas que necesito dado una fila que en realidad representa un QuestionGrid.
En efecto con el siguiente ejemplo:
import collections
df = pd.DataFrame({"QID":[1177],"Questions":["The travel restrictions of COVID-19 have been lifted and you are looking to book a flight. To what extent are the following factors considerations in your choice of flight?"],"QType":["Likert Scale"],"Answer0":["Very important consideration"],"Answer1":["Important consideration"],"Answer2":["Somewhat consider"],"Answer3":["Not an important consideration"],"Answer4":["Do not consider"],"Answer5":["Discounted flights"],"Answer6":["Very important consideration"],"Answer7":["Important consideration"],"Answer8":["Somewhat consider"],"Answer9":["Not an important consideration"],"Answer10":["Do not consider"],"Answer11":["Baggage policy"],"Answer12":["Very important consideration"],"Answer13":["Important consideration"],"Answer14":["Somewhat consider"],"Answer15":["Not an important consideration"],"Answer16":["Do not consider"],"Answer17":["Price of flights"],"Answer18":["Very important consideration"],"Answer19":["Important consideration"],"Answer20":["Somewhat consider"],"Answer21":["Not an important consideration"],"Answer22":["Do not consider"],"Answer23":["Insurance"],"Answer24":["Very important consideration"],"Answer25":["Important consideration"],"Answer26":["Somewhat consider"],"Answer27":["Not an important consideration"],"Answer28":["Do not consider"],"Answer29":["Airport services"],"Answer30":["Very important consideration"],"Answer31":["Important consideration"],"Answer32":["Somewhat consider"],"Answer33":["Not an important consideration"],"Answer34":["Do not consider"],"Answer35":["Environmental impact"],"Answer36":["Very important consideration"],"Answer37":["Important consideration"],"Answer38":["Somewhat consider"],"Answer39":["Not an important consideration"],"Answer40":["Do not consider"],"Answer41":["In-flight service"],"Answer42":["Very important consideration"],"Answer43":["Important consideration"],"Answer44":["Somewhat consider"],"Answer45":["Not an important consideration"],"Answer46":["Do not consider"],"Answer47":["Customer support"],"Answer48":["Very important consideration"],"Answer49":["Important consideration"],"Answer50":["Somewhat consider"],"Answer51":["Not an important consideration"],"Answer52":["Do not consider"],"Answer53":["Overcrowding on aircraft/airports"],"Answer54":["Very important consideration"],"Answer55":["Important consideration"],"Answer56":["Somewhat consider"],"Answer57":["Not an important consideration"],"Answer58":["Do not consider"],"Answer59":["Airport safety after COVID-19"],"Answer60":["Very important consideration"],"Answer61":["Important consideration"],"Answer62":["Somewhat consider"],"Answer63":["Not an important consideration"],"Answer64":["Do not consider"],"Answer65":["Refund policy"]})
def getquestions(r):
repeat = list({k:v for k,v in collections.Counter(r[3:].values).items() if v>1})
questions = []
firstfound = 0
for i in range(3, len(r)-len(repeat)):
if r[i:i+len(repeat)].tolist()==repeat:
if r[i+len(repeat):i+len(repeat)+1].values[0] is not None:
questions.append(r[i+len(repeat):i+len(repeat)+1].values[0])
if firstfound==0: firstfound = i+len(repeat)
if len(questions) > 0:
# somethong odd, sometimes it's a list other times a str
newq = r[1] + questions if isinstance(r[1], list) else [r[1]] + questions
r[1] = newq
# reset all the questions that have been used by list
for i in range(firstfound, len(r)):
if isinstance(r[i], str): r[i] = None
return r
def fixqid(c):
return [id if i==0 or c[i-1]!=id else f"{id}_{i}" for i, id in enumerate(c)]
df = df.apply(lambda r: getquestions(r), axis=1).explode("Questions").reset_index().drop("index", 1)
df["QID"] = fixqid(df["QID"].values)
df
Obtengo:
QID Questions QType Answer0 Answer1 Answer2 Answer3 Answer4 Answer5 Answer6 ... Answer56 Answer57 Answer58 Answer59 Answer60 Answer61 Answer62 Answer63 Answer64 Answer65
0 1177 The travel restrictions of COVID-19 have been ... Likert Scale Very important consideration Important consideration Somewhat consider Not an important consideration Do not consider None None ... None None None None None None None None None None
1 1177_1 Discounted flights Likert Scale Very important consideration Important consideration Somewhat consider Not an important consideration Do not consider None None ... None None None None None None None None None None
2 1177_2 Baggage policy Likert Scale Very important consideration Important consideration Somewhat consider Not an important consideration Do not consider None None ... None None None None None None None None None None
3 1177_3 Price of flights Likert Scale Very important consideration Important consideration Somewhat consider Not an important consideration Do not consider None None ... None None None None None None None None None None
4 1177_4 Insurance Likert Scale Very important consideration Important consideration Somewhat consider Not an important consideration Do not consider None None ... None None None None None None None None None None
5 1177_5 Airport services Likert Scale Very important consideration Important consideration Somewhat consider Not an important consideration Do not consider None None ... None None None None None None None None None None
6 1177_6 Environmental impact Likert Scale Very important consideration Important consideration Somewhat consider Not an important consideration Do not consider None None ... None None None None None None None None None None
7 1177_7 In-flight service Likert Scale Very important consideration Important consideration Somewhat consider Not an important consideration Do not consider None None ... None None None None None None None None None None
8 1177_8 Customer support Likert Scale Very important consideration Important consideration Somewhat consider Not an important consideration Do not consider None None ... None None None None None None None None None None
9 1177_9 Overcrowding on aircraft/airports Likert Scale Very important consideration Important consideration Somewhat consider Not an important consideration Do not consider None None ... None None None None None None None None None None
10 1177_10 Airport safety after COVID-19 Likert Scale Very important consideration Important consideration Somewhat consider Not an important consideration Do not consider None None ... None None None None None None None None None None
11 1177_11 Refund policy Likert Scale Very important consideration Important consideration Somewhat consider Not an important consideration Do not consider None None ... None None None
Entonces hoy querio aplicarlo a cada lineas de la dataframe.
for i, row in df.iterrows():
passed_items = []
for cell in row:
if cell in passed_items:
print("Line representing a Question Grid Detected")
df_to_insert = getquestions(row)
for j in range(len(df_to_insert)):
df.loc[i+j] = df_to_insert.loc[i]
else:
passed_items.append(str(cell))
Pero obtengo:
Line representing a Question Grid Detected
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-188-ea807caf35a2> in <module>
6 df_to_insert = getquestions(row)
7 for j in range(len(df_to_insert)):
----> 8 df.loc[i+j] = df_to_insert.loc[i]
9 else:
10 passed_items.append(str(cell))
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexing.py in __getitem__(self, key)
1766
1767 maybe_callable = com.apply_if_callable(key, self.obj)
-> 1768 return self._getitem_axis(maybe_callable, axis=axis)
1769
1770 def _is_scalar_access(self, key: Tuple):
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexing.py in _getitem_axis(self, key, axis)
1962
1963 # fall thru to straight lookup
-> 1964 self._validate_key(key, axis)
1965 return self._get_label(key, axis=axis)
1966
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexing.py in _validate_key(self, key, axis)
1829
1830 if not is_list_like_indexer(key):
-> 1831 self._convert_scalar_indexer(key, axis)
1832
1833 def _is_scalar_access(self, key: Tuple) -> bool:
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexing.py in _convert_scalar_indexer(self, key, axis)
739 ax = self.obj._get_axis(min(axis, self.ndim - 1))
740 # a scalar
--> 741 return ax._convert_scalar_indexer(key, kind=self.name)
742
743 def _convert_slice_indexer(self, key: slice, axis: int):
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexes\base.py in _convert_scalar_indexer(self, key, kind)
2886 elif kind in ["loc"] and is_integer(key):
2887 if not self.holds_integer():
-> 2888 self._invalid_indexer("label", key)
2889
2890 return key
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexes\base.py in _invalid_indexer(self, form, key)
3075 """
3076 raise TypeError(
-> 3077 f"cannot do {form} indexing on {type(self)} with these "
3078 f"indexers [{key}] of {type(key)}"
3079 )
TypeError: cannot do label indexing on <class 'pandas.core.indexes.base.Index'> with these indexers [4] of <class 'int'>
Qaui el problema es que una fila que no representa un QuestionGrid fue detectado como esto.