0

Mi servidor MySQL desaparece después de una consulta:

(venv) C:\Users\antoi\Documents\Programming\Work\data-tools>python -m etl.main
2021-07-06 12:37:15.134728 - Connecting to database hozana_data...
2021-07-06 12:37:15.140731 - Connecting to archive database hozana_archive...
2021-07-06 12:37:15.372728 - Start ETL main process
2021-07-06 12:37:15.372728 - `users` table:
2021-07-06 12:37:15.372728 - Hashing column `users`.`email:`
table_name:  users
c.execute("SELECT * FROM users"):  0
 done.
2021-07-06 12:37:15.374727 - Hashing column `users`.`email_notification:`
table_name:  users
(2006, '')
2021-07-06 12:37:15.375740 - Hashing column `users`.`last_name:`
table_name:  users
(2006, '')
2021-07-06 12:37:15.376728 - `messages` table:
Traceback (most recent call last):
  File "C:\Users\antoi\Documents\Programming\Work\data-tools\etl\task\anonymization.py", line 71, in main
    c.execute("""UPDATE messages SET payload = '' WHERE type = 'text_message'""")
  File "C:\Users\antoi\Documents\Programming\Work\data-tools\venv\lib\site-packages\MySQLdb\cursors.py", line 183, in execute
    while self.nextset():
  File "C:\Users\antoi\Documents\Programming\Work\data-tools\venv\lib\site-packages\MySQLdb\cursors.py", line 137, in nextset
    nr = db.next_result()
MySQLdb._exceptions.OperationalError: (2006, '')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\antoi\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\antoi\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\antoi\Documents\Programming\Work\data-tools\etl\main.py", line 52, in <module>
    main()
  File "C:\Users\antoi\Documents\Programming\Work\data-tools\etl\main.py", line 24, in main
    anonymization.main()
  File "C:\Users\antoi\Documents\Programming\Work\data-tools\etl\task\anonymization.py", line 71, in main
    c.execute("""UPDATE messages SET payload = '' WHERE type = 'text_message'""")
  File "C:\Users\antoi\Documents\Programming\Work\data-tools\venv\lib\site-packages\MySQLdb\connections.py", line 239, in __exit__
    self.close()
MySQLdb._exceptions.OperationalError: (2006, '')

Aqui esta el codigo:

def hash_column(table_name, column_name, pk_name, email_mode=False):
    # Executed a query that will overwrite a column with an hashed version of its content.
    # With email mode, the domain name will be kept:
    # [email protected] -> 379ac32fe8f576c4c63b17cd576e6c40c7dcd[...][email protected]
        try:
            info('Hashing column `{table_name}`.`{column_name}:`'
                 .format(table_name=table_name, column_name=column_name), end='', flush=True)
            with db as c:
                # Execute the update in batches, to avoid "Lock wait timeout exceeded"
                print("\ntable_name: ", table_name)
                c = c.cursor()
                print("c.execute(\"""SELECT * FROM users\"""): ", c.execute("SELECT * FROM users"))
                c.execute("""
                    SELECT {pk_name} as row_id
                    FROM {table_name}
                    WHERE {column_name} IS NOT NULL AND {column_name} NOT LIKE 'hash_%'
                    ORDER BY row_id ASC
                """.format(table_name=table_name, column_name=column_name, pk_name=pk_name))
                ids = []
                for row in c.fetchall():
                    row_id, = row
                    # cast to string, to avoid "expected str instance, int found" on join
                    ids.append(str(row_id))

                ids_chunks = chunks(list(ids), 50000)

                for ids in ids_chunks:
                    expression = """
                            CONCAT(
                                SHA2(SUBSTRING_INDEX(LOWER({column_name}), '@', 1), 512),
                                '@',
                                SUBSTRING_INDEX(LOWER({column_name}), '@', -1)
                            )
                        """.format(column_name=column_name) if email_mode \
                        else 'SHA2(LOWER({column_name}), 512)'.format(column_name=column_name)
                    print("c.execute(\"""SELECT COUNT(*) FROM users\"""): ",
                          c.execute("""SELECT COUNT(*) FROM {table_name}""").format(table_name=table_name))
                    c.execute("""
                        UPDATE `{table_name}`
                        SET `{column_name}` = CONCAT('hash_', {expression})
                        WHERE {pk_name} IN ({ids})
                    """.format(
                        table_name=table_name, column_name=column_name, expression=expression, pk_name=pk_name,
                        ids=','.join(ids)
                    ))

                    print('.', end='', flush=True)

            print(' done.')
        except MySQLdb.OperationalError as e:
            print(e)

Parece que es cuando el cliente no puede enviar una pregunta al servidor, muy probablemente porque el propio servidor ha cerrado la conexión. Sin embargo pensaba que ya habia la conexion:

introducir la descripción de la imagen aquí

Ademas hay estas lineas al principio:

2021-06-29 10:59:37.814133 - Connecting to database hozana_data...
2021-06-29 10:59:37.822142 - Connecting to archive database hozana_archive...

Intenté aumentar el max_allowed_packet:

mysql> select @@max_allowed_packet;
+----------------------+
| @@max_allowed_packet |
+----------------------+
|              4194304 |
+----------------------+
1 row in set (0.00 sec)

mysql> set global max_allowed_packet=10485760;
Query OK, 0 rows affected (0.01 sec)

Pero sigue siendo el mismo espacio:

mysql> select @@max_allowed_packet;
+----------------------+
| @@max_allowed_packet |
+----------------------+
|              4194304 |
+----------------------+
1 row in set (0.00 sec)
8
  • ¿Son muchos los datos que estás actualizando? Es probable que el servidor se cierre por demasiados datos. Si puedes configurarlo aumenta max_allowed_packet, y si no tienes acceso trata de hacer la operación en varios pasos.
    – Juan
    Commented el 29 jun. 2021 a las 12:52
  • ¡0, @Juan ! Es sólo un esquema, no hay datos. Commented el 29 jun. 2021 a las 13:26
  • Si no estas actualizando muchos datos el problema no es ese parámetro . El servidor cierra la conexión es lo único claro. Doy por hecho que has probado directamente desde la consola de MySQL yno solo desde programa.
    – Juan
    Commented el 29 jun. 2021 a las 16:21
  • @Juan Si, exactamente Commented el 30 jun. 2021 a las 8:31
  • Trabajando desde la consola de MySQL, ¿obtienes el mismo error al hacer un select en esta tabla de tu pregunta? ¿y que ocurre con otras tablas? Lo digo por aislar el error y asegurarse, ver si se debe a la tabla concreta o al servidor
    – Juan
    Commented el 30 jun. 2021 a las 8:49

0

Tu Respuesta

By clicking “Publica tu respuesta”, you agree to our terms of service and acknowledge you have read our privacy policy.

Examina otras preguntas con la etiqueta o formula tu propia pregunta.