2

Tengo un dataframe que contiene noticias para cada dia y intento analizar la intensidad del sentimiento por la dia, es por decir si el sentimiento general del día a partir de la noticia es positivo, negativo o neutral. Aqui esta la dataframe df_news:

    Date    name
0   2017-10-20  Gucci debuts art installation at its Ginza sto...
1   2018-08-01  Gucci Joins Paris Fashion Week for Its Spring ...
2   2018-04-20  Gucci launches its new creative hub Gucci ArtL...
3   2017-10-20  Gucci to launch homeware line Gucci Decor - CP...
4   2017-12-07  GUCCI opens new store at Miami Design District...
5   2018-01-12  Gucci opens Gucci Garden in Florence - LUXUO
6   2018-02-26  GUCCI's wild experiment with the Fall Winter 2...
7   2018-08-09  Gucci Revamped London Flagship Store | The Imp...
8   2018-08-01  Alessandro Michele Announces new Gucci Home co...
9   2017-10-20  Before He Picks Up the CFDA’s International Aw...

Intenté obtener la intensidad del sentimiento con el siguiente código que utiliza SentimentIntensityAnalyzer de nltk.sentiment.vader :

from nltk.sentiment.vader import SentimentIntensityAnalyzer
import unicodedata
sid = SentimentIntensityAnalyzer()
for date, row in df_news.T.iteritems():
    try:
        sentence = unicodedata.normalize('NFKD', df_news.loc[date, 'name']).encode('ascii','ignore')
        #print((sentence))
        ss = sid.polarity_scores(str(sentence))
        df_news.set_value(date, 'compound', ss['compound'])
        df_news.set_value(date, 'neg', ss['neg'])
        df_news.set_value(date, 'neu', ss['neu'])
        df_news.set_value(date, 'pos', ss['pos'])
    except TypeError:
        print(df_news.loc[date, 'name'])
        print(date)

Sin embargo obtengo una TypeError par ciertas fechas. Gracias al try catch no lo tiene en cuenta y me dibuja la siguiente tabla :

    name    compound    neg neu pos
Date                    
2017-10-20  Gucci debuts art installation at its Ginza sto...               
2018-08-01  Gucci Joins Paris Fashion Week for Its Spring ...               
2018-04-20  Gucci launches its new creative hub Gucci ArtL...   0.4404  0   0.756   0.244
2017-10-20  Gucci to launch homeware line Gucci Decor - CP...               
2017-12-07  GUCCI opens new store at Miami Design District...   0   0   1   0
2018-01-12  Gucci opens Gucci Garden in Florence - LUXUO    0   0   1   0
2018-02-26  GUCCI's wild experiment with the Fall Winter 2...   0   0   1   0
2018-08-09  Gucci Revamped London Flagship Store | The Imp...   0.3182  0   0.602   0.398
2018-08-01  Alessandro Michele Announces new Gucci Home co...               
2017-10-20  Before He Picks Up the CFDA’s International Aw...               

Pero cuando elimino try catch para entender por qué falló, recibo el siguiente error :

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-26-2e9dbfc62bce> in <module>
      4 for date, row in df_news.T.iteritems():
      5 #    try:
----> 6     sentence = unicodedata.normalize('NFKD', df_news.loc[date, 'name']).encode('ascii','ignore')
      7     #print((sentence))
      8     ss = sid.polarity_scores(str(sentence))

TypeError: normalize() argument 2 must be str, not Series

Entonces pensé que el problema fue con las líneas que no fueron string pero por ejemplo con la primera

>>>type(df_news['name'][0])
str

Para obtener las noticias

doc_data = {
  "size": 10,
  "query": {
    "bool": {
      "must" : [
       {"term":{"text":"gucci"}}
     ]
    }
  }
 }

docs = create_doc("https://elastic:[email protected]/luxurynsight_v2/news/_search",doc_data)


information_df = pd.DataFrame.from_dict(docs.json()["hits"]["hits"])

# Reading the JSON file
df_news = pd.read_json('data.json')

# Converting the element wise _source feature datatype to dictionary
df_news._source = df_news._source.apply(lambda x: dict(x))

# Creating name column
df_news['name'] = df_news._source.apply(lambda x: x['name'])

# Creating createdAt column
df_news['createdAt'] = df_news._source.apply(lambda x: x['createdAt'])

df_news['createdAt'] =  pd.to_datetime(df_news['createdAt'], unit='ms')

df_news['createdAt'] = pd.DatetimeIndex(df_news.createdAt).normalize()
#df_news.createdAt.dt.normalize()

df_news['Date'] = df_news['createdAt']

df_news = df_news[['name','Date']]
df_news = df_news.set_index('Date')
information_df._source = information_df.apply(lambda x: dict(x))
df_news.reset_index()

Y le da :

    Date    name
0   2017-10-20  Gucci debuts art installation at its Ginza sto...
1   2018-08-01  Gucci Joins Paris Fashion Week for Its Spring ...
2   2018-04-20  Gucci launches its new creative hub Gucci ArtL...
3   2017-10-20  Gucci to launch homeware line Gucci Decor - CP...
4   2017-12-07  GUCCI opens new store at Miami Design District...
5   2018-01-12  Gucci opens Gucci Garden in Florence - LUXUO
6   2018-02-26  GUCCI's wild experiment with the Fall Winter 2...
7   2018-08-09  Gucci Revamped London Flagship Store | The Imp...
8   2018-08-01  Alessandro Michele Announces new Gucci Home co...
9   2017-10-20  Before He Picks Up the CFDA’s International Aw...

0

Tu Respuesta

By clicking “Publica tu respuesta”, you agree to our terms of service and acknowledge you have read our privacy policy.

Examina otras preguntas con la etiqueta o formula tu propia pregunta.