Twitter Analysis of Vaccination in a Post COVID-19 Era

class: center, middle, inverse, title-slide

# Twitter Analysis of Vaccination in a Post COVID-19 Era
### <div class="line-block"> 
Nathalie de Marcellis-Warin, PhD (Polytechnique Montréal, CIRANO & OBVIA) 
Thierry Warin, PhD (HEC Montréal, CIRANO & OBVIA)</div>

---

## Agenda

1. Introduction

2. Data Collection

3. Methodology

4. Results and lessons learned

5. Conclusion

---
class: inverse, center, middle

# 1. Introduction

---

## 1. Introduction

> Twitter Analysis of Vaccination in a Post COVID-19 Era

- Our purpose is above all methodological and relates to the feasibility and validity of the analysis of conversations in a context of great importance for humanity, that of the Covid-19 disease.

- A **Data Science**-based use case

---

## 1. Introduction

> RQ1: Can we identify phases in conversations on COVID-19 on social media?

> RQ2: Do the conversations follow traditional media information?

- "communications" / "conversations"

- vertical / horizontal communications

- **Unstructured real-time data**: collecting live tweets + New York times articles

- **Natural Language Processing** for text analysis

---

## 1. Introduction

- We collected more than 2 million tweets and performed pattern detection

- We collected articles from the New York times corresponding to the same patterns (cosine similarity)

---

## 1. Introduction

- **Structured and unstructured data**

- Textual data are ubiquitous in social science research: traditional media, social media, survey data, and many other sources contribute to the enormous amount of text in the modern information age.

- **Structural Topic Model** (STM)

- The **increasing availabilit**y and interest in **textual data** has led to the development of a variety of statistical approaches to analyze these data.

---

## 1. Introduction

- With STM, users can model framing of **international newspapers** (Roberts, Stewart, and Airoldi 2016b), open-ended **survey responses** in the American National Election Study (Robertset al., 2014), **online classroom** forums (Reich, Tingley, Leder -Luis, Roberts, and Stewart 2015), **religious statements** (Lucas, Nielsen, Roberts, Stewart, Storer, and Tingley 2015), and **lobbying reports** (Milner and Tingley 2015), and so on.

- The purpose is to allow researchers **to discover topics** and estimate their **relationship to document metadata**.

- The results of the model can be used to perform hypothesis testing on these relationships.

- This, of course, mirrors the type of analysis that social scientists perform with other types of data, where the goal is to discover relationships between variables and test hypotheses.

---
class: inverse, center, middle

# 2. Data collection

---

## Temporal Distribution of Tweets

.panelset[

.panel[.panel-name[Overall Period]

.pull-left[

* **Period:** January 01, 2018 to April 21, 2021

* **Number of tweets** 2,601,702

* **Keywords:** “vaccine” OR “vaccines” OR “vaccinate” OR “vaccination” OR “vaccineswork” OR “antivax” OR “vaccinesdontwork” OR “provax” OR “vaxwithme” OR “antivaxxers” OR “immunization”

]

.pull-right[

]
]

.panel[.panel-name[Pandemic Period]

.pull-left[

* **Period:** January 01, 2020 to April 21, 2021

* **Number of tweets** 2,409,522

]

.pull-right[

]
]

.panel[.panel-name[Part 1]

.pull-left[

* **Period:** January 01, 2020 to September 30, 2020

* **Number of tweets:** 374,084

* **Proportion:** 15,5% of total number of tweets

]

.pull-right[

]
]

.panel[.panel-name[Part 2]

.pull-left[

* **Period:** October 01, 2020 to April 21, 2021

* **Number of tweets:** 2,035,438

* **Proportion:** 84,5% of total number of tweets

]

.pull-right[

]

---
class: inverse, center, middle

# 3 Methodology

---

## 3.1 Overall Time Series

.panelset[

.panel[.panel-name[Method]

**Event study**

- We used an **Event study** approach.

- In order to perform this type of analysis, financial specialists use the return, volume or volatility of stocks. It is about quantifying the economic impact of an event in so-called abnormal returns. For our study, we chose to use the number of tweets per day to detect **anomalies**.

]

.panel[.panel-name[Step 1]

.pull-left[

- We performed an anomaly detection on all tweets from the beginning of the pandemic (January 01, 2020 to April 21, 2021), which corresponds to 2,409,522 tweets.

- We noticed an acceleration of the conversation on Twitter about vaccination starting on October 1, 2020. So we decided to split the tweets into two parts:
  - Part 1: January 01, 2020 to September 30, 2020
    - Number of tweets: 374,084
  - Part 2: October 01, 2020 to April 21, 2021
    - Number of tweets: 2,035,438
]

.pull-right[

]
]

.panel[.panel-name[Step 2]

.pull-left[

For each part we performed a new anomaly detection.

* **Part 1 (01-01-2020 to 30-09-2020):** Anomaly performed with the following parameters:
  * alpha = 0.05 or 5% of outlier data
  * max_anoms = 0.2 or 20% of anomalies allowed.

* **Part 2 (01-10-2020 to 21-04-2021):** Anomaly performed with adjusted parameters: 
    * alpha = 0.3
    * max_anoms = 0.05
    
]

.pull-right[

]
    
]

.panel[.panel-name[Step 3]

.pull-left[

* We have grouped the anomalies by sub periods. 
  * **Part 1:** We have identified 5 sub periods
  * **Part 2:** We have observed 6 sub periods.

* Time interval of each sub Period: 
  * **Start of the intverval:** date 1st anomaly - 3 days (period_start) 
  * **End of the interval:** last anomaly date + 3 days (period_end)

]

.pull-right[

]
]

.panel[.panel-name[Summary of Periods]

.pull-left[

<template id="4c3676ba-aecd-4092-9984-6da2b88f48a0"><style>
.tabwid table{
 border-spacing:0px !important;
 border-collapse:collapse;
 line-height:1;
 margin-left:auto;
 margin-right:auto;
 border-width: 0;
 display: table;
 margin-top: 1.275em;
 margin-bottom: 1.275em;
 border-color: transparent;
}
.tabwid_left table{
 margin-left:0;
}
.tabwid_right table{
 margin-right:0;
}
.tabwid td {
 padding: 0;
}
.tabwid a {
 text-decoration: none;
}
.tabwid thead {
 background-color: transparent;
}
.tabwid tfoot {
 background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-bdc9f240{}.cl-bdc2cda8{font-family:'DejaVu Sans';font-size:8pt;font-weight:bold;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-bdc2cdbc{font-family:'DejaVu Sans';font-size:8pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-bdc2e450{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-bdc2e45a{margin:0;text-align:right;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-bdc3279e{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc327a8{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc327c6{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc327c7{width:48.5pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc327d0{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc327da{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc327e4{width:48.5pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc327e5{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc327f8{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc32802{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc32803{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc3280c{width:48.5pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc32816{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc32817{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc32820{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc3282a{width:48.5pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc32834{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc3283e{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc32848{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bdc32849{width:48.5pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-bdc9f240'><caption class="Table Caption">

Sub-periods of anomalies for the 1st part of the pandemic period

</caption><thead><tr style="overflow-wrap:break-word;"><td class="cl-bdc32834">Period</td><td class="cl-bdc32848">Start date</td><td class="cl-bdc3283e">End date</td><td class="cl-bdc32849">Number of tweets</td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-bdc3279e">Period 1</td><td class="cl-bdc327c6">2020-03-13</td><td class="cl-bdc327a8">2020-03-19</td><td class="cl-bdc327c7">10,581</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bdc327da">Period 2</td><td class="cl-bdc327e5">2020-05-12</td><td class="cl-bdc327d0">2020-05-21</td><td class="cl-bdc327e4">23,651</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bdc327da">Period 3</td><td class="cl-bdc327e5">2020-07-13</td><td class="cl-bdc327d0">2020-07-30</td><td class="cl-bdc327e4">41,797</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bdc327f8">Period 4</td><td class="cl-bdc32803">2020-08-08</td><td class="cl-bdc32802">2020-08-14</td><td class="cl-bdc3280c">15,415</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bdc32816">Period 5</td><td class="cl-bdc32820">2020-09-05</td><td class="cl-bdc32817">2020-09-19</td><td class="cl-bdc3282a">35,681</td></tr></tbody></table></div></template>
<div class="flextable-shadow-host" id="b7b94b96-27fb-4ee2-80ff-377a8de4695c"></div>
<script>
var dest = document.getElementById("b7b94b96-27fb-4ee2-80ff-377a8de4695c");
var template = document.getElementById("4c3676ba-aecd-4092-9984-6da2b88f48a0");
var caption = template.content.querySelector("caption");
if(caption) {
 caption.style.cssText = "display:block;text-align:center;";
 var newcapt = document.createElement("p");
 newcapt.appendChild(caption)
 dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

]

.pull-right[

<template id="9a9f8eec-5249-4bb9-a7b5-41b9d0cde752"><style>
.tabwid table{
 border-spacing:0px !important;
 border-collapse:collapse;
 line-height:1;
 margin-left:auto;
 margin-right:auto;
 border-width: 0;
 display: table;
 margin-top: 1.275em;
 margin-bottom: 1.275em;
 border-color: transparent;
}
.tabwid_left table{
 margin-left:0;
}
.tabwid_right table{
 margin-right:0;
}
.tabwid td {
 padding: 0;
}
.tabwid a {
 text-decoration: none;
}
.tabwid thead {
 background-color: transparent;
}
.tabwid tfoot {
 background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-bed44212{}.cl-beceb234{font-family:'DejaVu Sans';font-size:8pt;font-weight:bold;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-beceb248{font-family:'DejaVu Sans';font-size:8pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-becec0f8{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-becec10c{margin:0;text-align:right;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-becef640{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef654{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef65e{width:53.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef668{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef672{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef67c{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef686{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef690{width:53.6pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef69a{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef6a4{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef6ae{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef6b8{width:53.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef6b9{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef6c2{width:53.6pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef6cc{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-becef6cd{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-bed44212'><caption class="Table Caption">

Sub-periods of anomalies for the 2nd part of tweets

</caption><thead><tr style="overflow-wrap:break-word;"><td class="cl-becef6b9">Period</td><td class="cl-becef6cc">Start date</td><td class="cl-becef6cd">End date</td><td class="cl-becef6c2">Number of tweets</td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-becef654">Period 1</td><td class="cl-becef668">2020-11-06</td><td class="cl-becef640">2020-11-12</td><td class="cl-becef65e">38,704</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-becef686">Period 2</td><td class="cl-becef672">2020-11-23</td><td class="cl-becef67c">2020-11-29</td><td class="cl-becef690">35,851</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-becef686">Period 3</td><td class="cl-becef672">2020-11-29</td><td class="cl-becef67c">2020-12-17</td><td class="cl-becef690">211,684</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-becef686">Period 4</td><td class="cl-becef672">2020-12-21</td><td class="cl-becef67c">2021-01-04</td><td class="cl-becef690">124,038</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-becef686">Period 5</td><td class="cl-becef672">2021-02-27</td><td class="cl-becef67c">2021-03-05</td><td class="cl-becef690">105,605</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-becef6ae">Period 6</td><td class="cl-becef69a">2021-04-10</td><td class="cl-becef6a4">2021-04-16</td><td class="cl-becef6b8">98,570</td></tr></tbody></table></div></template>
<div class="flextable-shadow-host" id="d2cee367-53f9-405e-b2e0-7aa311aa1962"></div>
<script>
var dest = document.getElementById("d2cee367-53f9-405e-b2e0-7aa311aa1962");
var template = document.getElementById("9a9f8eec-5249-4bb9-a7b5-41b9d0cde752");
var caption = template.content.querySelector("caption");
if(caption) {
 caption.style.cssText = "display:block;text-align:center;";
 var newcapt = document.createElement("p");
 newcapt.appendChild(caption)
 dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

]
]

]

---

## 3.2 Thematic Model

.panelset[

.panel[.panel-name[Method]

.pull-left[

**Structural Topic Models (STM)**

- The goal of the STM is to allow researchers to discover topics and estimate their relationship to document metadata.

- The following figure presents a heuristic overview of a typical workflow.

- First users ingest the data and prepare it for analysis. Next a structural topic model is estimated. The ability to estimate the structural topic model quickly allows for the evaluation, understanding, and visualization of results [@roberts_stm_2019]

]

.pull-right[

<div id="htmlwidget-1ec9256a3dfd2730bec6" style="width:70%;height:504px;" class="grViz html-widget"></div>
<script type="application/json" data-for="htmlwidget-1ec9256a3dfd2730bec6">{"x":{"diagram":"\ndigraph boxes_and_circles {\n\n # a \"graph\" statement\n graph [overlap = true, fontsize = 8, compound = true, nodesep = .5, ranksep = .5,\n color = black]\n\n # several \"node\" statements\n node [shape = box,\n fontname = Helvetica,\n fixedsize = false,\n shape = rectangle, \n width = 1,\n color = black,\n fontcolor = black]\n edge [arrowsize = 0.6, color = black]\n\n # several \"edge\" statements\n \"INGEST \nReading and processing \ntext data\"->\"PREPARE \nAssociating text with \nmetadata\"->\"ESTIMATE \nEstimating the structural \ntopic model\"\n \"ESTIMATE \nEstimating the structural \ntopic model\"->\"EVALUATE \nModel selection \nand search\" \n \"ESTIMATE \nEstimating the structural \ntopic model\"->\"UNDERSTAND \nInterpreting the STM \nby plotting \nand inspecting results\" \n \"ESTIMATE \nEstimating the structural \ntopic model\"->\"VISUALIZE \nPresenting STM results\"\n \"EXTENSIONS \nAdditional tools for \ninterpretation and \nvisualization\"\n }\n}\n","config":{"engine":"dot","options":null}},"evals":[],"jsHooks":[]}</script>

]

.panel[.panel-name[Process]

We performed an STM model to determine the top 10 topics for:

- the part 1 of tweets (January 1, 2020 to September 30, 2020.),

- the part 2 of tweets (October 1, 2020 to April 21, 2021),

- each of the 11 sub periods.

]

---

## 3.2 Thematic Model

.panelset[

.panel[.panel-name[Pre Covid-19 Era]

Period: 2018-01-01 to 2019-11-30

]

.panel[.panel-name[Post Covid-19 Era]

Period: 2019-12-01 to 2021-04-21

]
]

---

## 3.2 Thematic Model

.panelset[

.panel[.panel-name[P1 P1: Top 10 topics]

.pull-left[

Part 1 Period 1 (March 13-19, 2020)

]

.pull-right[

]
]

.panel[.panel-name[P1 P1: Wordclouds]

Part 1 Period 1 (March 13-19, 2020). Topic 8 & 9 respectively.

]

.panel[.panel-name[P2 P6: Top 10 topics]

.pull-left[

Part 2 Period 6 (April 10-16, 2021)

]

.pull-right[

]
]

.panel[.panel-name[P2 P6: Wordclouds]

Part 2 Period 6 (April 10-16, 2021). Topic 7 & 6 respectively.

]
]

---

## 3.2 Thematic Model

.panelset[

.panel[.panel-name[Summary of topics]

**Top 3 topics for each 11 sub-period**

.pull-left[

<table class=" lightable-classic" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto;'>
 <thead>
 <tr>
 <th style="text-align:left;"> Date </th>
 <th style="text-align:left;"> Topic </th>
 </tr>
 </thead>
<tbody>
 <tr>
 <td style="text-align:left;"> 13-03-2020 to 19-03-2020 </td>
 <td style="text-align:left;"> covid, develop, amp; peopl, get, can; trial, coronavirus, test </td>
 </tr>
 <tr>
 <td style="text-align:left;"> 12-05-2020 to 21-05-2020 </td>
 <td style="text-align:left;"> get, peopl, need; covid, coronavirus, research; trump, coronavirus, presid </td>
 </tr>
 <tr>
 <td style="text-align:left;"> 13-07-2020 to 13-07-2020 </td>
 <td style="text-align:left;"> get, peopl, like; amp, need, work; covid, trial, phase </td>
 </tr>
 <tr>
 <td style="text-align:left;"> 08-08-2020 to 14-08-2020 </td>
 <td style="text-align:left;"> get, peopl, like; russia, first, coronavirus; covid, coronavirus, dose </td>
 </tr>
 <tr>
 <td style="text-align:left;"> 05-09-2020 to 19-09-2020 </td>
 <td style="text-align:left;"> trump, say, coronavirus; coronavirus, covid, first; covid, develop, work </td>
 </tr>
</tbody>
</table>

]

.pull-right[

<table class=" lightable-classic" style='font-family: "Arial Narrow", "Source Sans Pro", sans-serif; margin-left: auto; margin-right: auto;'>
 <thead>
 <tr>
 <th style="text-align:left;"> Date </th>
 <th style="text-align:left;"> Topic </th>
 </tr>
 </thead>
<tbody>
 <tr>
 <td style="text-align:left;"> 06-11-2020 to 12-011-2020 </td>
 <td style="text-align:left;"> get, peopl, flu; pfizer, covid, effect; news, stock, hope </td>
 </tr>
 <tr>
 <td style="text-align:left;"> 23-11-2020 to 29-11-2020 </td>
 <td style="text-align:left;"> covid, effect, astrazeneca; take, time, now; covid, develop, india </td>
 </tr>
 <tr>
 <td style="text-align:left;"> 29-11-2020 to 17-12-2020 </td>
 <td style="text-align:left;"> peopl, get, need; first, covid, dose; pfizer, approv, covid </td>
 </tr>
 <tr>
 <td style="text-align:left;"> 21-12-2020 to 04-01-2021 </td>
 <td style="text-align:left;"> covid, dose, first; covid, health, state; get, take, like </td>
 </tr>
 <tr>
 <td style="text-align:left;"> 26-02-2021 to 05-03-2021 </td>
 <td style="text-align:left;"> effect, like, just; covid, johnson, dose; covid, get, can </td>
 </tr>
 <tr>
 <td style="text-align:left;"> 10-04-2021 to 16-04-2021 </td>
 <td style="text-align:left;"> peopl, just, make; johnson, amp, paus; dose, receiv, million </td>
 </tr>
</tbody>
</table>

]
]
]

---

## 3.3 Cosine similarity

.panelset[

.panel[.panel-name[Method]

- Document similarity (or distance between documents) is one of the central themes in Information Retrieval. How humans usually define how similar are documents? Usually documents treated as similar if they are semantically close and describe similar concepts. [@selivanov_dselivanovtext2vec_2021]

Classical approach from computational linguistics is to measure similarity based on the content overlap between documents. For this documents are presented as bag-of-words, so each document will be a sparse vector. And measure of overlap will be defined as angle between vectors:

$$
\text{similarity}\left(doc_1,doc_2\right)=\cos\left(\theta\right)=\frac{doc_1doc_2}{\left|doc_1\right|\left|doc_2\right|}
$$

By cosine distance/dissimilarity we assume following:

$$
\text{distance}\left(doc_1,doc_2\right)=1-\text{similarity}\left(doc_1,doc_2\right)
$$

It is important to note, however, that this is not a proper distance metric in a mathematical sense as it does not have the triangle inequality property and it violates the coincidence axiom.

]

.panel[.panel-name[NYT articles]

.pull-left[

We calculated the cosine similarity between the tweets of each 11 sub periods and the New York Times (NYT) articles for the same periods.

First, we collected all the NYT articles using the dates of each sub period (see the following table).

]

.pull-right[

<template id="ae0ffdd3-b43a-4933-b027-3a600cba934f"><style>
.tabwid table{
 border-spacing:0px !important;
 border-collapse:collapse;
 line-height:1;
 margin-left:auto;
 margin-right:auto;
 border-width: 0;
 display: table;
 margin-top: 1.275em;
 margin-bottom: 1.275em;
 border-color: transparent;
}
.tabwid_left table{
 margin-left:0;
}
.tabwid_right table{
 margin-right:0;
}
.tabwid td {
 padding: 0;
}
.tabwid a {
 text-decoration: none;
}
.tabwid thead {
 background-color: transparent;
}
.tabwid tfoot {
 background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-bfbd506a{}.cl-bfb79aa8{font-family:'DejaVu Sans';font-size:8pt;font-weight:bold;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-bfb79abc{font-family:'DejaVu Sans';font-size:8pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-bfb7a7dc{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-bfb7a7e6{margin:0;text-align:right;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-bfb7ddb0{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7ddba{width:50.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7ddc4{width:44.3pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7ddce{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7ddd8{width:50.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7ddd9{width:44.3pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7ddec{width:50.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7dded{width:44.3pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7ddf6{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7de00{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7de01{width:50.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7de0a{width:44.3pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7de0b{width:44.3pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7de28{width:50.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7de29{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bfb7de32{width:44.3pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-bfbd506a'><caption class="Table Caption">

</caption><thead><tr style="overflow-wrap:break-word;"><td class="cl-bfb7de32">Part</td><td class="cl-bfb7de29">Period</td><td class="cl-bfb7de28">Nb. of articles</td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td rowspan="5"class="cl-bfb7ddc4">Part 1</td><td class="cl-bfb7ddb0">Period 1</td><td class="cl-bfb7ddba">1,149</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bfb7de00">Period 2</td><td class="cl-bfb7de01">1,462</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bfb7de00">Period 3</td><td class="cl-bfb7de01">2,485</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bfb7ddce">Period 4</td><td class="cl-bfb7ddd8">879</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bfb7de00">Period 5</td><td class="cl-bfb7de01">1,997</td></tr><tr style="overflow-wrap:break-word;"><td rowspan="6"class="cl-bfb7de0b">Part 2</td><td class="cl-bfb7de00">Period 1</td><td class="cl-bfb7de01">1,058</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bfb7ddce">Period 2</td><td class="cl-bfb7ddd8">829</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bfb7de00">Period 3</td><td class="cl-bfb7de01">2,563</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bfb7de00">Period 4</td><td class="cl-bfb7de01">1,459</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bfb7ddce">Period 5</td><td class="cl-bfb7ddd8">945</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bfb7ddf6">Period 6</td><td class="cl-bfb7ddec">991</td></tr></tbody></table></div></template>
<div class="flextable-shadow-host" id="a97450dc-468d-4666-a614-a1a6ebe27085"></div>
<script>
var dest = document.getElementById("a97450dc-468d-4666-a614-a1a6ebe27085");
var template = document.getElementById("ae0ffdd3-b43a-4933-b027-3a600cba934f");
var caption = template.content.querySelector("caption");
if(caption) {
 caption.style.cssText = "display:block;text-align:center;";
 var newcapt = document.createElement("p");
 newcapt.appendChild(caption)
 dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

]

.panel[.panel-name[NYT articles about vaccine]

.pull-left[

Second, we filtered the collected NYT articles per sub periods to only keep those that contained at least one of the following words:

- “vaccine” OR “vaccines” OR “vaccinate” OR “vaccination” OR “vaccineswork” OR “antivax” OR “vaccinesdontwork” OR “provax” OR “vaxwithme” OR “antivaxxers” OR “immunization”.

The following table display the number of articles remaining after the filtering.

]

.pull-right[

<template id="ef8dab18-5d03-4748-b701-a5b56dc29cf5"><style>
.tabwid table{
 border-spacing:0px !important;
 border-collapse:collapse;
 line-height:1;
 margin-left:auto;
 margin-right:auto;
 border-width: 0;
 display: table;
 margin-top: 1.275em;
 margin-bottom: 1.275em;
 border-color: transparent;
}
.tabwid_left table{
 margin-left:0;
}
.tabwid_right table{
 margin-right:0;
}
.tabwid td {
 padding: 0;
}
.tabwid a {
 text-decoration: none;
}
.tabwid thead {
 background-color: transparent;
}
.tabwid tfoot {
 background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
</style><div class="tabwid"><style>.cl-c004c9d6{}.cl-bffe8544{font-family:'DejaVu Sans';font-size:8pt;font-weight:bold;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-bffe8562{font-family:'DejaVu Sans';font-size:8pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-bffe93d6{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-bffe93e0{margin:0;text-align:right;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-bffed7a6{width:50.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed7b0{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed7ba{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed7c4{width:44.3pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed7c5{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed7ce{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed7cf{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed7d8{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed7e2{width:50.4pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed7e3{width:44.3pt;background-color:transparent;vertical-align: middle;border-bottom: 0.5pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed7f6{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed7f7{width:50.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed80a{width:44.3pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed814{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed828{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0.5pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed832{width:67pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed83c{width:53.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed846{width:68.7pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed850{width:50.4pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-bffed851{width:44.3pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-c004c9d6'><caption class="Table Caption">

</caption><thead><tr style="overflow-wrap:break-word;"><td class="cl-bffed851">Part</td><td class="cl-bffed83c">Period</td><td class="cl-bffed846">Start</td><td class="cl-bffed832">End</td><td class="cl-bffed850">Nb. of articles</td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td rowspan="5"class="cl-bffed7c4">Part 1</td><td class="cl-bffed7b0">Period 1</td><td class="cl-bffed7ba">2020-03-13</td><td class="cl-bffed7c5">2020-03-19</td><td class="cl-bffed7a6">4</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bffed7ce">Period 2</td><td class="cl-bffed7cf">2020-05-12</td><td class="cl-bffed7d8">2020-05-21</td><td class="cl-bffed7e2">18</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bffed7ce">Period 3</td><td class="cl-bffed7cf">2020-07-13</td><td class="cl-bffed7d8">2020-07-30</td><td class="cl-bffed7e2">28</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bffed7ce">Period 4</td><td class="cl-bffed7cf">2020-08-08</td><td class="cl-bffed7d8">2020-08-14</td><td class="cl-bffed7e2">9</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bffed7ce">Period 5</td><td class="cl-bffed7cf">2020-09-05</td><td class="cl-bffed7d8">2020-09-19</td><td class="cl-bffed7e2">44</td></tr><tr style="overflow-wrap:break-word;"><td rowspan="6"class="cl-bffed80a">Part 2</td><td class="cl-bffed7ce">Period 1</td><td class="cl-bffed7cf">2020-11-06</td><td class="cl-bffed7d8">2020-11-12</td><td class="cl-bffed7e2">14</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bffed7ce">Period 2</td><td class="cl-bffed7cf">2020-11-23</td><td class="cl-bffed7d8">2020-11-29</td><td class="cl-bffed7e2">21</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bffed7ce">Period 3</td><td class="cl-bffed7cf">2020-11-29</td><td class="cl-bffed7d8">2020-12-17</td><td class="cl-bffed7e2">163</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bffed7ce">Period 4</td><td class="cl-bffed7cf">2020-12-21</td><td class="cl-bffed7d8">2021-01-04</td><td class="cl-bffed7e2">66</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bffed7ce">Period 5</td><td class="cl-bffed7cf">2021-02-27</td><td class="cl-bffed7d8">2021-03-05</td><td class="cl-bffed7e2">50</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-bffed7f6">Period 6</td><td class="cl-bffed828">2021-04-10</td><td class="cl-bffed814">2021-04-16</td><td class="cl-bffed7f7">56</td></tr></tbody></table></div></template>
<div class="flextable-shadow-host" id="7ed4b00a-e2c7-4625-9f54-042ec6e767ff"></div>
<script>
var dest = document.getElementById("7ed4b00a-e2c7-4625-9f54-042ec6e767ff");
var template = document.getElementById("ef8dab18-5d03-4748-b701-a5b56dc29cf5");
var caption = template.content.querySelector("caption");
if(caption) {
 caption.style.cssText = "display:block;text-align:center;";
 var newcapt = document.createElement("p");
 newcapt.appendChild(caption)
 dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

]
]

.panel[.panel-name[Mean cosine similarity]

]
]

---
class: inverse, center, middle

# 4. Results and Lessons

---

## Results and Lessons

- During the first 5 periods, the conversations were chronologically about:
  
  - the **fears**: are we going to have a vaccine?
  - then on the **debate** on vaccines
  - then on the **economy**, the relationship between vaccines and the return to work
  - then on the invention of the vaccine and its **political** dimension
  - finally on the **economy** again as a reason to be vaccinated
  
- During the last 6 periods, the conversations allowed the level of development of epidemiological expertise:
 
  - the **hope** dimension: Pfizer
  - then **AstraZeneca** and a developing country dimension
  - then Pfizer to be prioritized over AstraZeneca
  - then on the importance of getting vaccinated anyway
  - then a conversation about vaccines again when the vaccination campaigns started
  - the question of which vaccine is better.
  
  
---

## Results and Lessons

1. Lessons for **research**:
  
  - proof of concept on the quantitative measurement of conversation dynamics on a topic like SARS-CoV-II
  - reflections on how to use these conversation dynamics on social networks as evidence
  - multidisciplinary: linguistics, communications, economics, public policy

2. Lessons for **public policy** in particular:
  
  - reflections on the use of these social network data transformation methods for the relevance of public policy communications
  - national and temporal measures: possibility to build a national barometer to know the dynamics of conversations on a particular public policy topic.

---
class: inverse, center, middle

# 5. Conclusion

---

## Conclusion

> RQ1: Can we identify phases in conversations on COVID-19 on social media?

> RQ2: Do the conversations follow traditional media information?

- **No evidence** of high similarity between traditional media and social media on these conversations

- Some room for **optimism**:

- the horizontal conversations have become **more educated** about the pandemic, though political debates could not be avoided at some point => COVID-19 fatigue?

---

## Conclusion

- **Unstructured data** from two sources: **Twitter** and **New York Times**

- **NLP**: natural language analysis protocol

- **Nowcasting** for public policy communications

---
class: inverse, middle, center

# Thanks!