0

Buenas tardes, intento importar un csv file en Hbase que esta muy largo. Esta tratando de los productos alimenticios: ingredientes, nutrición, etiquetas. Proviene de Open Food Facts. Enumera información sobre productos alimenticios: ingredientes, información nutricional, etiquetas, etc. La mayoría de los datos proviene de la información de crowdsourcing. El archivo esta sobre de la plataforma abierta de datos públicos franceses.

He visto este comando para un archivo csv con 2 columnas :

hbase org.apache.hadoop.hbase.mapreduce.ImportTsv –Dimporttsv.columns=HBASE_ROW_KEY,cf1:name,cf2:exp bulktable /hbase/bulk_data.tsv

Pero yo tengo más de 150 columnas. Mer parece dificil copiar todas en este comando. ¿Hay alguna forma más sencilla de importar mi gran archivo csv en mi base de datos HBase?

Intento lo siguiente :

hbase(main):005:0> hadoop jar /usr/lib/hbase/lib/hbase-server-1.0.0-cdh5.4.4.jar 
importtsv '-Dimporttsv.separator=,' 
-Dimporttsv.bulk.output=output 
-Dimporttsv.columns=HBASE_ROW_KEY,code,f: url,f: creator,f: created_t,f: created_datetime, ...
...
f:water-hardness_100g store /bulkload/perftest/bouffe.csv

Pero me contesta el terminal :

SyntaxError: (hbase):4: lbunknown regexp options - lb

El comando completa esta :

hbase(main):005:0> hadoop jar /usr/lib/hbase/lib/hbase-server-1.0.0-cdh5.4.4.jar importtsv '-Dimporttsv.separator=,' -Dimporttsv.bulk.output=output -Dimporttsv.columns=HBASE_ROW_KEY,code,f: url,f: creator,f: created_t,f: created_datetime,f: last_modified_t,f: last_modified_datetime,f: product_name,f: generic_name,f: quantity,f: packaging,f: packaging_tags,f: brands,f: brands_tags,f: categories,f: categories_tags,f: categories_fr,f: origins,f: origins_tags,f: manufacturing_places,f: manufacturing_places_tags,f: labels,f: labels_tags,f: labels_fr,f: emb_codes,f: emb_codes_tags,f: first_packaging_code_geo,f: cities,f: cities_tags,f: purchase_places,f: stores,f: countries,f: countries_tags,f: countries_fr,f: ingredients_text,f: allergens,f: allergens_fr,f: traces,f: traces_tags,f: traces_fr,f: serving_size,f: no_nutriments,f: additives_n,f: additives,f: additives_tags,f: additives_fr,f: ingredients_from_palm_oil_n,f: ingredients_from_palm_oil,f: ingredients_from_palm_oil_tags,f: ingredients_that_may_be_from_palm_oil_n,f: ingredients_that_may_be_from_palm_oil,f: ingredients_that_may_be_from_palm_oil_tags,f: nutrition_grade_uk,f: nutrition_grade_fr,f: pnns_groups_1,f: pnns_groups_2,f: states,f: states_tags,f: states_fr,f: main_category,f: main_category_fr,f: image_url,f: image_small_url,f: energy_100g,f: energy-from-fat_100g,f: fat_100g,f: saturated-fat_100g,f: butyric-acid_100g,f: caproic-acid_100g,f: caprylic-acid_100g,f: capric-acid_100g,f: lauric-acid_100g,f: myristic-acid_100g,f: palmitic-acid_100g,f: stearic-acid_100g,f: arachidic-acid_100g,f: behenic-acid_100g,f: lignoceric-acid_100g,f: cerotic-acid_100g,f: montanic-acid_100g,f: melissic-acid_100g,f: monounsaturated-fat_100g,f: polyunsaturated-fat_100g,f: omega-3-fat_100g,f: alpha-linolenic-acid_100g,f: eicosapentaenoic-acid_100g,f: docosahexaenoic-acid_100g,f: omega-6-fat_100g,f: linoleic-acid_100g,f: arachidonic-acid_100g,f: gamma-linolenic-acid_100g,f: dihomo-gamma-linolenic-acid_100g,f: omega-9-fat_100g,f: oleic-acid_100g,f: elaidic-acid_100g,f: gondoic-acid_100g,f: mead-acid_100g,f: erucic-acid_100g,f: nervonic-acid_100g,f: trans-fat_100g,f: cholesterol_100g,f: carbohydrates_100g,f: sugars_100g,f: sucrose_100g,f: glucose_100g,f: fructose_100g,f: lactose_100g,f: maltose_100g,f: maltodextrins_100g,f: starch_100g,f: polyols_100g,f: fiber_100g,f: proteins_100g,f: casein_100g,f: serum-proteins_100g,f: nucleotides_100g,f: salt_100g,f: sodium_100g,f: alcohol_100g,f: vitamin-a_100g,f: beta-carotene_100g,f: vitamin-d_100g,f: vitamin-e_100g,f: vitamin-k_100g,f: vitamin-c_100g,f: vitamin-b1_100g,f: vitamin-b2_100g,f: vitamin-pp_100g,f: vitamin-b6_100g,f: vitamin-b9_100g,f: folates_100g,f: vitamin-b12_100g,f: biotin_100g,f: pantothenic-acid_100g,f: silica_100g,f: bicarbonate_100g,f: potassium_100g,f: chloride_100g,f: calcium_100g,f: phosphorus_100g,f: iron_100g,f: magnesium_100g,f: zinc_100g,f: copper_100g,f: manganese_100g,f: fluoride_100g,f: selenium_100g,f: chromium_100g,f: molybdenum_100g,f: iodine_100g,f: caffeine_100g,f: taurine_100g,f: ph_100g,f: fruits-vegetables-nuts_100g,f: fruits-vegetables-nuts-estimate_100g,f: collagen-meat-protein-ratio_100g,f: cocoa_100g,f: chlorophyl_100g,f: carbon-footprint_100g,f: nutrition-score-fr_100g,f: nutrition-score-uk_100g,f: glycemic-index_100g,f: water-hardness_100gcode, f:url, f:creator, f:created_t, f:created_datetime, f:last_modified_t, f:last_modified_datetime, f:product_name, f:generic_name, f:quantity, f:packaging, f:packaging_tags, f:brands, f:brands_tags, f:categories, f:categories_tags, f:categories_fr, f:origins, f:origins_tags, f:manufacturing_places, f:manufacturing_places_tags, f:labels, f:labels_tags, f:labels_fr, f:emb_codes, f:emb_codes_tags, f:first_packaging_code_geo, f:cities, f:cities_tags, f:purchase_places, f:stores, f:countries, f:countries_tags, f:countries_fr, f:ingredients_text, f:allergens, f:allergens_fr, f:traces, f:traces_tags, f:traces_fr, f:serving_size, f:no_nutriments, f:additives_n, f:additives, f:additives_tags, f:additives_fr, f:ingredients_from_palm_oil_n, f:ingredients_from_palm_oil, f:ingredients_from_palm_oil_tags, f:ingredients_that_may_be_from_palm_oil_n, f:ingredients_that_may_be_from_palm_oil, f:ingredients_that_may_be_from_palm_oil_tags, f:nutrition_grade_uk, f:nutrition_grade_fr, f:pnns_groups_1, f:pnns_groups_2, f:states, f:states_tags, f:states_fr, f:main_category, f:main_category_fr, f:image_url, f:image_small_url, f:energy_100g, f:energy-from-fat_100g, f:fat_100g, f:saturated-fat_100g, f:butyric-acid_100g, f:caproic-acid_100g, f:caprylic-acid_100g, f:capric-acid_100g, f:lauric-acid_100g, f:myristic-acid_100g, f:palmitic-acid_100g, f:stearic-acid_100g, f:arachidic-acid_100g, f:behenic-acid_100g, f:lignoceric-acid_100g, f:cerotic-acid_100g, f:montanic-acid_100g, f:melissic-acid_100g, f:monounsaturated-fat_100g, f:polyunsaturated-fat_100g, f:omega-3-fat_100g, f:alpha-linolenic-acid_100g, f:eicosapentaenoic-acid_100g, f:docosahexaenoic-acid_100g, f:omega-6-fat_100g, f:linoleic-acid_100g, f:arachidonic-acid_100g, f:gamma-linolenic-acid_100g, f:dihomo-gamma-linolenic-acid_100g, f:omega-9-fat_100g, f:oleic-acid_100g, f:elaidic-acid_100g, f:gondoic-acid_100g, f:mead-acid_100g, f:erucic-acid_100g, f:nervonic-acid_100g, f:trans-fat_100g, f:cholesterol_100g, f:carbohydrates_100g, f:sugars_100g, f:sucrose_100g, f:glucose_100g, f:fructose_100g, f:lactose_100g, f:maltose_100g, f:maltodextrins_100g, f:starch_100g, f:polyols_100g, f:fiber_100g, f:proteins_100g, f:casein_100g, f:serum-proteins_100g, f:nucleotides_100g, f:salt_100g, f:sodium_100g, f:alcohol_100g, f:vitamin-a_100g, f:beta-carotene_100g, f:vitamin-d_100g, f:vitamin-e_100g, f:vitamin-k_100g, f:vitamin-c_100g, f:vitamin-b1_100g, f:vitamin-b2_100g, f:vitamin-pp_100g, f:vitamin-b6_100g, f:vitamin-b9_100g, f:folates_100g, f:vitamin-b12_100g, f:biotin_100g, f:pantothenic-acid_100g, f:silica_100g, f:bicarbonate_100g, f:potassium_100g, f:chloride_100g, f:calcium_100g, f:phosphorus_100g, f:iron_100g, f:magnesium_100g, f:zinc_100g, f:copper_100g, f:manganese_100g, f:fluoride_100g, f:selenium_100g, f:chromium_100g, f:molybdenum_100g, f:iodine_100g, f:caffeine_100g, f:taurine_100g, f:ph_100g, f:fruits-vegetables-nuts_100g, f:fruits-vegetables-nuts-estimate_100g, f:collagen-meat-protein-ratio_100g, f:cocoa_100g, f:chlorophyl_100g, f:carbon-footprint_100g, f:nutrition-score-fr_100g, f:nutrition-score-uk_100g, f:glycemic-index_100g, f:water-hardness_100g store /bulkload/perftest/bouffe.csv

1 respuesta 1

0

Recuerda que HBASE trabaja por familias de columnas, así que debes crear primero tu base. Si persiste el error, leí en un foro en inglés que puedes intentar ejecutando fuera del shell de hbase.

También puedes intentar migrando primero tu CSV a TSV

Te comparto el link de la documentación oficial de HBase para carga masiva.

1
  • 1
    Bienvenido a Stack Overflow en Español, tu respuesta puede intentar solucionar la pregunta, pero deberías incluir alguna explicación que ayude a entender lo que propones (principalmente para la comunidad) e incluir algún ejemplo de cómo se puede aplicar lo que propones. Te recomiendo que leas el siguiente artículo: ¿Cómo escribo una buena respuesta?. Commented el 18 may. 2018 a las 15:57

Tu Respuesta

By clicking “Publica tu respuesta”, you agree to our terms of service and acknowledge you have read our privacy policy.

¿No es la respuesta que buscas? Examina otras preguntas con la etiqueta o formula tu propia pregunta.