BLOG

Valgkampens tredje uge er forbi, og vi er gået ind i den sidste spændende uge inden valget den 5. juni. Inden da tager Digital Media Lab et overblik over den foregående uge. I dette tilfælde strækker ugen sig fra den 21. maj og til og med den 27. maj 2019.

I denne udgave kigger vi, ligesom sidste uge, på følgende ting:

• Hvilken profil har lavet flest tweets?
•  Hvilke hashtags har været mest populære?
• Hvilken profil er blevet nævn flest gange?
• Hvilken profil har over hele valgkampens tweetet flest gange?

I forlængelse med ovenstående kigger vi samtidig også lidt nærmere på Europa-Parlamentsvalgets indflydelse på folketingsvalgkampen.

Lad os for god ordens skyld kigge lidt nærmere på hvilke profiler der har lavet flest tweets i den forgangene uge.

Antal tweets
I alt er der blevet lavet 6.660 tweets fordelt ud over 560 kandidater. Dette inkluderer også partiernes egne Twitter profiler. Top fem over mest tweetede profiler ser således ud:

1. Karen Melchior – Kommende Europa-Parlaments medlem for de Radikale (758 tweets)
2. Uffe Elbæk – Formand og medlem af partiet Alternativet (193 tweets)
3. Uwe Max Jensen – Folketingskandidat for Strams Kurs i Nordjylland (172 tweets)
4. Andreas Albertsen – Folketingskandidat for SF i Vestjylland (148 tweets)
5. Niels Callesøe – Folketingskandidat for SF i Østjylland (147 tweets).

Det er i dette sammenhæng værd at nævne, at retweets indgår i det samlede antal tweets. Karen Melchior er stadigvæk aktiv på Twitter, men de fleste af hendes tweets er retweets. Karen Melchior, som i øvrigt for nyligt blev valgt ind i Europa Parlamentet, sprænger altså grafen med hendes 758 tweets. For at sætte det antal i relief, så svarer det til at hun per time laver 4,5 tweets i syv fulde dage.

Nedenfor ses søjlediagrammet over dette:

Hashtags
Som skrevet tidligere, så har der i denne periode været 6.660 tweets, og emnerne de berører, er meget ens. For god ordens skyld så skal vi igen nævne, at de nedenstående grafer er foruden hashtaggene #dkpol, #fv19 og #dkmedier. I denne udgave har vi lavet to grafer. En der indeholder hashtagget #ep19dk og en der ikke gør. Det har vi gjort for at kunne vise emnerne, så det er mere realistisk hvad emner har været. Vi skal i den forbindelse også nævne, at valgkampshashtag som #folketingsvalg19 #fv2019 #valg2019 er blevet lagt sammen, så alle tal indgår. Det vil sige, at placeringerne af hashtags i grafen ikke nødvendigt stemmer overens med deres reelle placering.

Rækkefølgen er:

1. #ep19dk (brugt 510 gange)
2. #dkgreen (brugt 168 gange)
3. #fremad (brugt 135 gange)
4. #viereuropa (brugt 90 gange)
5. #stemgrønnest (brugt 66 gange)
6. #alternativet (brugt 51 gange)
7. #grøntvalg (brugt 41 gange)
8. #hopeisback (brugt 31 gange)
9. #sundpol (brugt 29 gange)
10. #Klimamarch (brugt 23 gange)

Grafen med hashtagget #ep19dk ser således ud:

Grafen uden hashtagget #ep19dk ser således ud:

Mentions
I de to sidste uger af valgkampen, har det været Socialdemokratiets Twitter profil (Spolitik) der er blevet nævnt flest gange. I valgkampens tredje uge må de se sig slået af Alternativets Twitter profil (alternativet_). Rækkefølgen er følgende:

1. Alternativets twitterprofil (alternativet_) er nævnt 295 gange
2. Socialdemokratiets twitterprofil (Spolitk) er nævnt 244 gange
3. Venstres twitterprofil (venstredk) er nævnt 191 gange
4. Det Radikale Venstres twitterprofil (radikale) er nævnt 176 gange
5. Karen Melchiors twitterprofil (karmel80) er nævnt 166 gange

Længere nede på listen finder man partiformænd og kvinder som Uffe Elbæk, Lars Løkke Rasmussen, Pia Kjærsgaard og Pia Olsen Dyhr. Grafen ser således ud:

Samlet tweets
Tre ugers valgkamp har været undervejs, og den sidste uge er næsten kun lige begyndt. I den forbindelse har Digital Media Lab taget et dyk ned i hvem der har tweetet mest. På nuværende tidspunkt kan det næppe komme som nogen overraskelse, at det er kommende medlem af Europa Parlamentet for de Radikale Karen Melchior, der er på en stensikker førsteplads. Med 1361 tweets, har hun lavet dobbelt så mange tweets som andenpladsen. Andenpladsen er besat af Uwe Max Jensen som er folketingskandidat for Stram Kurs i Nordjylland.

Rækkefølgende er:

1. Karen Melchior – Kommende Europa-Parlaments medlem for de Radikale (1361 tweets)
2. Uwe Max Jensen – Folketingskandidat for Strams Kurs i Nordjylland (562 tweets)
3. Andreas Albertsen – Folketingskandidat for SF i Vestjylland (551 tweets)
4. Uffe Elbæk – Formand og medlem af partiet Alternativet (508 tweets)
5. Niels Callesøe – Folketingskandidat for SF i Østjylland (486 tweets).

Grafen ser således ud:

Dette overblik er lavet af forskningsassistent Nicolaj Sveiger i samarbejde med adjunkt Sander Andreas Schwartz. 

We are currently attending the European Investigative Journalism Conference (EIJC19) in Mechelen, and will write about interesting software usable for research purposes.

This list of software is something we took a note of, and that you might find interesting.

• Datashare – a cool piece of software for reading documents and turn unstructured data into structures.
• Neo4j – Graph software, that turns relational data into just that – relations, that can be visualized in many ways. Using a language quite similar to SQL. Syntax is pretty complex though.
• Anaconda – Once again the Anaconda platform seems to be the weapon of choice for coders around the world.
• OSINT Framework – Framework focused on gathering information from free tools or resources. Also on Github.
• Python Package Index – Great overview of libraries. Searchable, obviously.

And as a small bonus, we did a bit of coding while at the conference. So we have updated our scripts for converting handles and id’s for Twitter users (and vice versa). Now they output both to command line and CSV-files 🙂

Have you ever encountered the following?

You have read a bunch of documents. Think a lot of documents. But after reading, you are thinking of a specific detail. A detail you have forgotten. The name of a company or product for instance. And really, you are not interested in reading all the documents again. So you start scimming the texts for the missing detail.

Good news – now you won’t have to scim it your self. Let Datashare do it for you.

Will Fitzgibbon of ICIJ has written a pretty good guide for Datashare, you could start by reading.

ICIJ is the International Consortium of Investigative Journalists. The organization behind the Panama Papers, Lux Leaks, Off shore leaks, Implant files and many other large scale international collaborative (data) journalism projects.

We are currently attending the European Investigative Journalism Conference (EIJC19) in Mechelen, and will write about interesting software usable for research purposes.

Link: Datashare on ICIJ
Link: Dataharvest/EIJC

If you are interested in collecting and analyzing data from Facebook pages we hereby provide a short how-to guide. You can only collect data from public pages so that means no automated collection of groups or personal profiles. You might experience some issues with posts that are not collected. Always check that the data you collect appears to be complete by going to the facebook page and comparing numbers of likes/comments/shares and posts. I you experience any issue, then try to collect smaller amounts of data in a shorter amount of time. You can always merge multiple data sets later on.

This guide requires Excel and a Facebook account.

1. Find Facebook ID
   • Go to the Facebook page that you want to collect data from via a web browser
   • Copy the URL address
   • Paste the address into the available space at https://lookup-id.com/
   • Copy the number (Facebook ID) that you receive through this process.
2. Collect data from Facebook pages via Netvizz
   • Go to the app Netvizz via url: https://apps.facebook.com/107036545989762/
   • Install and accept permission for the app. This is a university app developed by the University of Amsterdam, and your data is not stored on their servers.
   • Press the link Page posts and insert the FB ID
   • Choose date and other collection details. Leave the posts statistics only selected unless you are specifically interested in the content of comments
   • Press posts by page only or posts by page and users
   • Scroll down to the bottom bellow both graphs. This is tricky on a Mac, since you might not see the option or scrolling. Bellow the two graphs you will find the download link saying Download link as zip file. Press that link to download the file.
3. Open in Excel
   • Unpack the zip-file by double pressing it.
   • Choose the file that is NOT called statsperday.
   • Open the the file by right clicking and going to open with click and other.
   • Select all programs rather than recommended programs and then find your Excel program.
   • Click the Excel program and view the data via Excel.

You should now have an overview of the collected data via Excel sorted into columns and rows. If the data does not appear to be sorted into columns then select column A and press Data and then text to columns. Choose delimited and than select tab and press finish.

Watch two how to guides from the developer himself. The interface has changed a little since then, but overall it is very similar.

Link to YouTube video

 

Ever wondered how scraping works? If you are pretty much blank when it comes to programming, this guide is probably not for you. However, if you have the basic concepts in place, in a few steps the author, Mikko Helsig, shows you how to scrape a site in Python (and also how to install Python in a Windows environment).

Prerequisites are a basic understanding of programming. But then you get a concept of how Python works, how scraping works and the really cool libraries requests, requests_cache, BeautifulSoup and Gender (the latter is a library used to guessing and parsing gender of names).

Link: Scraping 101

If you are totally new to programming, we encourage you to start by learning a little Python. There are numerous places to do this. For instance:

• Codecademy (both free and $)
• Learn Python (free)
• W3Schools Python Tutorial (free)

We also deeply encourage you to start with programming in an environment such as Anaconda. A short description of the Anaconda Navigator can be found here.

We have added Instaloader to our External resources.

Instaloader is a tool to download pictures (or videos) along with their captions and other metadata from Instagram. You can either download profiles or hashtags, and it’s possible to set up filters (for instance datefilters, see below) to narrow your search.

To use Instaloader,  you should do the following.

1. Download and set up a new Anaconda Environment with a Python version higher than 3.5.
2. Install Jyputer Notebook on the environment and open a new terminal
3. Do a pip (not pip3, as that does not work with Anaconda) install of the instaloader and dependencies

   pip install instaloader

4. Create a new folder in your root-environment (typically documents-folder) called for instance Instaloader
5. In terminal do

   cd instaloader

6. This is to avoid that everything is saved in your base folder 🙂
7. Run various command line commands in your terminal. Please do note that the interface is rudimentary but filters can be applied with the use of boolean expressions for instance:

   instaloader "#HASHTAG" --post-filter="date_utc >= datetime(2017,1,1) and date_utc <= datetime(2018,1,1)" --login=USERNAME

8. We would love to implement this as a hosted service. However, it is not likely we will do so just now. Therefore, please experiment with it yourself. You can also ask our advice, and we will do our best to help. If you plan to use this tool on a regular basis or for larger datasets, you should probably be ready to use several user accounts and/or proxies to avoid being banned.

On 28/11 2018 we had our official launch with a seminar and get-together at our place.

We had a smashing time discussing the possibilities with you guys, and look forward to realizing ideas from the day. If you would like to relive the day (and who wouldn’t), you will find links to presentations and programme below.

• Opening presentation by Jannie Møller Hartley
• Presentation by Sander Schwartz
• Presentation by Mads Kæmsgaard Eberholst
• Presentation by Eva Mayerhöffer
• Launch Programme

Our hosted service for collecting Twitter data called Yourtwapperkeeper is currently retired and is having a lovely time at the old software home.

Currently the data can still be downloaded, but the server will be completely closed in January 2019. If you are using data from that server, you should download it by then.

If you need Twitter-data collection please use our TCAT-servers instead.

All data and ongoing collections from Yourtwapperkeeper has already been migrated to TCAT. Read more about the TCAT services on our Hosted resources.

I det mange tilfælde oplever vi, at studerende og forskere helst vil tage kontakt med os via e-mail, men i efterårssemesteret 2018 eksperimenterer vi med begrænset åbningstid.

Indenfor den begrænsede åbningstid vil Sander sidde på kontoret. Du kan således benytte lejligheden til at komme uanmeldt, men ofte kan det være en god idé at skrive inden, så Sander ved, at du kommer og eventuelt kan forberede sig på at løse dit problem.

Åbningstiderne er i øjeblikket hveranden onsdag i perioden 13-15. Det drejer sig således om følgende datoer i 2018:

• 24/10, kl. 13-15
• 07/11, kl. 13-15
• 21/11, kl. 13-15
• 05/12, kl. 13-15
• 19/12, kl. 13-15