Estoy enviando los datos para los INDICES DE ELASTICSEARCH y en mi logstash configuro el .conf el cual le indico un query de SQL y todo parece ser correcto luego voy a KIBANA (previamente abro el tunel ssh de elasticsearch) en el devtool para verificar el proceso de indexado de datos.
CONSULTA SQL
SELECT V1.product_id, category_id, category_name, fullname, image, relevance, color, title, brand, vitrina, price, model, weight, height, is_prime, sales_quantity,bestseller , rating FROM (SELECT random() , p.product_id, p.category_id,c.name as category_name,c.fullname,f.url as image,p.relevance, p.color, p.title,p.brand, CASE WHEN p.vitrina IS TRUE THEN 1 ELSE 0 END AS vitrina,p.price, p.model, p.weight, p.height, p.is_prime,0 AS sales_quantity,CASE WHEN p.bestseller IS NULL THEN 0 ELSE p.bestseller END AS bestseller,CASE WHEN p.rating IS NULL THEN 0 ELSE rating END AS rating FROM products p JOIN categories c on p.category_id = c.category_id JOIN files f on p.product_id = f.product_id WHERE f.main = 1 AND p.status = 1 AND package_weight <= 120 order by p.relevance asc) V1
IMAGEN DEL DEVTOOL
lo que esta señalado en rojo son los documentos pendientes por indexar, pero nunca terminan y no se donde ver cual es la razon.
MAPPING DEL INDICE
PUT productos_v0_busqueda
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "standard",
"filter": [
"lowercase",
"asciifolding",
"snowball"
]
}
}
}
},
"mappings": {
"properties": {
"@timestamp" : {
"type" : "date"
},
"@version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"bestseller" : {
"type" : "long"
},
"category_id" : {
"type" : "long"
},
"category_name" : {
"type" : "text",
"analyzer": "my_analyzer",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"fullname" : {
"type" : "text",
"analyzer": "my_analyzer",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"height" : {
"type" : "float"
},
"image" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"is_prime" : {
"type" : "long"
},
"model" : {
"type" : "text",
"analyzer": "my_analyzer",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"product_id" : {
"type" : "long"
},
"rating" : {
"type" : "long"
},
"relevance" : {
"type" : "long"
},
"store_id" : {
"type" : "long"
},
"vitrina" : {
"type" : "long"
},
"weight" : {
"type" : "float"
},
"sales_quantity": {
"type": "double"
},
"price": {
"type": "double"
},
"brand": {
"type": "text",
"analyzer": "my_analyzer",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"color": {
"type": "text",
"analyzer": "my_analyzer",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"title": {
"type": "text",
"analyzer": "my_analyzer"
},
"title_ots": {
"type": "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
},
"analyzer": "my_analyzer"
},
"cat_level_0": {
"type": "text",
"copy_to": "all_categories",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"cat_level_1": {
"type": "text",
"copy_to": "all_categories",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"cat_level_2": {
"type": "text",
"copy_to": "all_categories",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"cat_level_3": {
"type": "text",
"copy_to": "all_categories",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"cat_level_4": {
"type": "text",
"copy_to": "all_categories",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"cat_level_5": {
"type": "text",
"copy_to": "all_categories",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"all_categories": {
"type": "text",
"analyzer": "my_analyzer",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"status":{
"type" : "long"
},
"discount_percentage":{
"type" : "float"
},
"stock":{
"type" : "long"
}
}
}
}
Sin embargo este parece que no esta indexando todos los datos y deja unos cuantos en pendiente. ¿saben de que manera puedo ver porque quedaron pendientes? ya que en el log no me muestra ningun error ni nada similar que me permita identificar cual es la falla.