1

Estoy tratando de obtener las preguntas relacionadas de este cuestionario

from collections import defaultdict
import json

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import ElementClickInterceptedException

import time

driver = webdriver.Chrome(executable_path=r"C:\Programs\chromedriver.exe")
location = "https://docs.google.com/spreadsheets/d/1iLqEFRaHPYxpJKU05VXt3HUCQ2OQUAg8FfWlyFbvaXc/edit?usp=sharing"
location = "https://docs.google.com/forms/d/e/1FAIpQLSfzocEm6IEDKVzVGOlg8ijysWZyAvQur0NheJb_I_xozgKusA/viewform?usp=sf_link"

class_name = "freebirdFormviewerViewItemsItemItemTitle.exportItemTitle.freebirdCustomFont"
driver.get(location)
time.sleep(1.5)


questions = driver.find_element_by_class_name(class_name)
print(questions.text)

Pero yo obtengo solo el primero:

(scr_env) C:\Users\antoi\Documents\Programming\Scraping\Python>python3 questions_scraper.py

DevTools listening on ws://127.0.0.1:55562/devtools/browser/d30bdba2-39f2-4ec9-8aa9-e61439b474b5
[38340:34544:0527/173840.814:ERROR:browser_switcher_service.cc(238)] XXX Init()
How often did you fly before the Covid-19 epidemic?

1 respuesta 1

2

Debes usar find_elements_by_class_name y no find_element_by_class_name e iterar sobre la lista, no obstante, si quieres obtener todas las preguntas deberás hacer click en el botón Siguiente y volver a buscar la clase.

Si además quieres la respuestas se complica un poco:

  • Buscamos todos los contenedores con cada pregunta y sus respuestas.
  • Por cada contenedor:
    • Obtenemos la pregunta
    • Obtenemos las respuestas
  • Click en el botón Siguiente

Lo anterior lo repetimos mientras no se termine el formulario.

from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException


location = "https://docs.google.com/forms/d/e/1FAIpQLSfzocEm6IEDKVzVGOlg8ijysWZyAvQur0NheJb_I_xozgKusA/viewform?usp=sf_link"

data = {}

driver = webdriver.Firefox()
driver.get(location)

title = driver.find_element_by_class_name(
    "freebirdFormviewerViewHeaderTitleRow"
    ).text
data[title] = {}

while True:
    containers = driver.find_elements_by_class_name(
        "freebirdFormviewerViewNumberedItemContainer"
        )
    btns = driver.find_elements_by_css_selector(".appsMaterialWizButtonEl")
    if not btns:
        break
    for container in containers:
        try:
            question = container.find_element_by_class_name(
                "freebirdFormviewerViewItemsItemItemTitle"
                )
        except NoSuchElementException:
            continue
        responses = container.find_elements_by_class_name(
            "docssharedWizToggleLabeledLabelText"
            )
        data[title][question.text] = [response.text for response in responses]
    btns[-1].click()
driver.quit()

print(data)

El resultado es un diccionario con todas las preguntas y sus respuestas posibles:

{
    "Air Travel": {
        "How often did you fly before the Covid-19 epidemic?": [
            "Once per year",
            "Between two and five times per year",
            "More than five times per year"
        ],
        "What did you dislike the most about Pre-Covid 19 air travel?": [
            "Long waits at security",
            "Layovers",
            "Entertainment on flight",
            "Cramped seating on the flight",
            "Otro:"
        ],
        "Did you feel safe flying before the Covid-19 pandemic?": [
            "Yes",
            "No"
        ],
        "How long will you wait before flying again, after Covid-19?": [
            "30 Days",
            "30 - 90 Days",
            "90 - 180 Days",
            "More than 180 Days"
        ],
        "Will you fly for pleasure or only when absolutely necessary?": [
            "Pleasure",
            "Only When Absolutely Necessary"
        ],
        "Would a flight with a guaranteed empty seat between you and the person sitting next to you make you more comfortable?": [
            "Yes",
            "No"
        ],
        "What can we do to make you feel as safe as possible while flying with us?": [],
        "Will you be more interested in the cheapest flight or the airline that offers the highest level of protection from disease?": [
            "Cheapest Flight",
            "Level of Protection From Disease",
            "Mix of Both"
        ],
        "Would you take advantage of a new class of seating that offered improved social distancing during the flight as well as complimentary personal protection equipment?": [
            "Yes",
            "No"
        ],
        "Which of these options would you like to see the most of your next flight?": [
            "The latest movies",
            "Free hand sanitizer and wipes",
            "Free meal for longer flight or free drinks on shorter ones",
            "Scheduled check-in time so you can avoid waiting in a large crowd",
            "Otro:"
        ]
    }
}

Tu Respuesta

By clicking “Publica tu respuesta”, you agree to our terms of service and acknowledge you have read our privacy policy.

¿No es la respuesta que buscas? Examina otras preguntas con la etiqueta o formula tu propia pregunta.