Generalizing Question Answering System with Pre-trained Language Model Fine-tuning

D Su, Y Xu, GI Winata, P Xu, H Kim, Z Liu, P Fung - … of the 2nd Workshop on Machine …, 2019

… (2009)) and Common Crawl (Buck et al., 1https://github.com/mrqa/MRQA-SharedTask-2019 2014) for pre-training … 2014. N-gram counts and language models from the common crawl. In LREC, volume 2, page 4. Citeseer …


Generating Black-Box Adversarial Examples for Text Classifiers Using a Deep Reinforced Model

P Vijayaraghavan, D Roy - arXiv preprint arXiv:1909.07873, 2019

… These paraphrase datasets together contains text from various sources: Common Crawl, CzEng1.6, Europarl, News Commentary, Quora questions, and Twitter trending topic tweets. We do not use all the data for our pretraining …


Generating composite SQL queries from natural language

… of the questions. We decided to use the Common Crawl embedding that is trained on 42 billion tokens, consists of a vocabulary of 1.9 million tokens and embeds these tokens in the 300-dimensional vector space5. All the words …


Generating Language-Independent Neural Sentence Embeddings for Natural Language Classification Tasks

… [Rud17] At the time this thesis was written, there are Word Embeddings for more than 150 languages, trained on Common Crawl1 and Wikipedia, available. [Rud17] 1An open repository of web crawl data that can be …


Generic Web Content Extraction with Open-Source Software

… Because of the vastly increasing variety of corpora, text types and use cases, it becomes more and more difficult to assess the usefulness and appropriateness of certain web texts 1https://commoncrawl.org for given research objectives …


Geo-spatial text-mining from Twitter–a feature space analysis with a view toward building classification in urban regions

M Häberle, M Werner, XX Zhu - European Journal of Remote Sensing, 2019


Ghmerti at SemEval-2019 Task 6: A Deep Word-and Character-based Approach to Offensive Language Identification

E Doostmohammadi, H Sameti, A Saffar - … of the 13th International Workshop on …, 2019

… The indices include 256 of the most common characters, plus 0 for padding and 1 for un- known characters. 2. xw which is the embeddings of the words in the input tweet based on FastText's 600Btoken common crawl model (Mikolov et al., 2018) …


GLOSS: Generative Latent Optimization of Sentence Representations

SP Singh, A Fan, M Auli - arXiv preprint arXiv:1907.06385, 2019

… representations. This could be as simple as using a bag-of-words averaging of Glove (Pennington et al., 2014) word embeddings trained on a corpus such as CommonCrawl, which we re- fer to as Glove-BoW. Methods such …


Z Zhu, S Xu, M Qu, J Tang - arXiv preprint arXiv:1903.00757, 2019

Page 1. GraphVite: A High-Performance CPU-GPU Hybrid System for Node Embedding Zhaocheng Zhu Mila - Québec AI Institute Université de Montréal zhaocheng.zhu@ umontreal.ca Shizhen Xu Tsinghua University xsz12@mails.tsinghua.edu.cn …


Green AI

R Schwartz, J Dodge, NA Smith, O Etzioni - arXiv preprint arXiv:1907.10597, 2019

… For example, the June 2019 Common Crawl contains 242 TB of uncompressed data,12 so even simple filtering to extract usable text is difficult … 11https://opensource.google.com/ projects/open-images-dataset 12http://commoncrawl.org/2019/07 …


Grounded Response Generation Task at DSTC7

M Galley, C Brockett, X Gao, J Gao, B Dolan

… Turn 4 still pretty incredible , but quite a bit different that 10,000 meters . Table 1: Sample of the DSTC7 Sentence Generation data, which combines Reddit data (Turns 1-4) along with documents (extracted from Common Crawl) discussed in the conversations …


Happy Together: Learning and Understanding Appraisal From Natural Language

… language models (ULMFiT). Exploiting Simple GloVe Embeddings For the embedding layer, we obtain the 300-dimensional embedding vector for tokens using GloVe's Common Crawl pre-trained model [13]. GloVe embeddings …


HARE: a Flexible Highlighting Annotator for Ranking and Exploration

D Newman-Griffis, E Fosler-Lussier - arXiv preprint arXiv:1908.11302, 2019

… ated three commonly used benchmark embedding sets: word2vec skipgram (Mikolov et al., 2013) using GoogleNews,6 FastText skipgram with subword information on WikiNews,7 and GloVe (Pennington et al., 2014) on 840 …


HATEMINER at SemEval-2019 Task 5: Hate speech detection against Immigrants and Women in Twitter using a Multinomial Naive Bayes Classifier

N Chakravartula - Proceedings of the 13th International Workshop on …, 2019

… Word Embeddings: Glove840B - common crawl, GloveTwitter27B - twitter crawl (Pennington et al., 2014) and fasttext - common crawl (Mikolov et al., 2018) pre-trained word embeddings are used to analyze their impact on the classification …


HealthSuggestions: moving beyond the beta version

Page 1. FACULDADE DE ENGENHARIA DA UNIVERSIDADE DO PORTO Health Suggestions: Moving Beyond the Beta Version Paulo Miguel Pereira dos Santos Master in Informatics and Computing Engineering Supervisor …


Hierarchical Meta-Embeddings for Code-Switching Named Entity Recognition

GI Winata, Z Lin, J Shin, Z Liu, P Fung - arXiv preprint arXiv:1909.08504, 2019

… We use FastText word embeddings trained from Common Crawl and Wikipedia (Grave et al., 2018) for English (es), Spanish (es), including four Romance languages: Catalan (ca), Portuguese (pt), French (fr), Italian …


High Quality ELMo Embeddings for Seven Less-Resourced Languages

M Ulčar, M Robnik-Šikonja - arXiv preprint arXiv:1911.10049, 2019

… They used 20-million-words data randomly sampled from the raw text released by the CoNLL 2017 Shared Task - Automatically Annotated Raw Texts and Word Embeddings (Ginter et al., 2017), which is a combination of Wikipedia dump and common crawl …


Hitachi at MRP 2019: Unified Encoder-to-Biaffine Network for Cross-Framework Meaning Representation Parsing

Y Koreeda, G Morio, T Morishita, H Ozaki, K Yanai - arXiv preprint arXiv:1910.01299, 2019

… Named entity label Named entity (NE) recognition is applied to the input text (see Section 7.1). GloVe We use 300-dimensional GloVe (Pennington et al., 2014) pretrained on Common Crawl2 which are kept fixed during the training …


HMM, Is This Ethical? Predicting the Ethics of Reddit Life Protips

… large corpus of text. GloVe representations have been trained on several large datasets that are publicly available, including corpuses from Wikipedia, Gigaword, Twitter, and Common Crawl [4]. 3. Task Definition Our problem is …


How Decoding Strategies Affect the Verifiability of Generated Text

L Massarelli, F Petroni, A Piktus, M Ott, T Rocktäschel… - arXiv preprint arXiv …, 2019

… consisting of roughly 3 Billion Words; (iv) CC- NEWS, a de-duplicated subset of the English portion of the CommonCrawl news dataset (Nagel, 2016; Bakhtin et al., 2019; Liu et al., 2019a), which totals around 16 Billion words …


How to Ask Better Questions? A Large-Scale Multi-Domain Dataset for Rewriting Ill-Formed Questions

Z Chu, M Chen, J Chen, M Wang, K Gimpel, M Faruqui… - arXiv preprint arXiv …, 2019

… and En↔Fr. The English-German translation models are trained on WMT datasets, including News Commentary 13, Europarl v7, and Common Crawl, and evaluated on newstest2013 for early stopping. On the newstest2013 …


How Well Do Embedding Models Capture Non-compositionality? A View from Multiword Expressions

N Nandakumar, T Baldwin, B Salehi - Proceedings of the 3rd Workshop on Evaluating …, 2019

… It tokenises text at the character level. fastText We used the 300-dimensional fastText model pre-trained on Common Crawl and Wikipedia using CBOW (fastTextpre), as well as one trained over the same Wikipedia corpus4 us- ing skip-gram (fastText) …


Hybrid Rule-Based Model for Phishing URLs Detection

KS Adewole, AG Akintola, SA Salihu, N Faruk… - International Conference for …, 2019

N Faruk, RG Jimoh - … International Conference, iCETiC 2019, London, UK …, 2019

… 1. From this figure, data collected from different servers such as Yahoo, Alexa, Common Crawl, PhishTank and OpenPhish are preprocessed in order to extract meaningful features that can be used for categorizing phishing websites from legitimate ones …


Hybrid Words Representation for Airlines Sentiment Analysis

… GloVe uses ratios of co-occurrence probabilities. It is favourable to concatenate ELMo embeddings with traditional word embeddings. In this work, we have used pre-trained GloVe embedding (trained on 840 billion token from common crawl) of 300 dimensions …


Hyper: Distributed Cloud Processing for Large-Scale Deep Learning Tasks

D Buniatyan - arXiv preprint arXiv:1910.07172, 2019

… [4] MinIO high performance object storage server compatible with Amazon S3 API. https://github.com/minio/minio, 2018. [Online; accessed 31- May-2019]. [5] Common Crawl Dataset. https://commoncrawl.org, 2019. [Online; accessed 31-May-2019] …


Hyperparameter Tuning for Deep Learning in Natural Language Processing

… on the Common Crawl, one on 42 and the other on 840 billion tokens), FastText (Bojanowski et al., 2016), dependency based (Levy and Goldberg, 2014), and ELMo (Peters et al., 2018). As shown in Ta- ble 1, the Glove …


Identification Of Bot Accounts In Twitter Using 2D CNNs On User-generated Contents

M Polignano, MG de Pinto, P Lops, G Semeraro - 2019

… FastTextEmb)8: 300 dimensionality vectors, composed by a vocabulary of 2 million words and n-grams of the words, case sensitive and obtained from 600 billion of tokens trained on data crawled from generic Internet web pages by Common Crawl nonprofit organization; …


Identification of Good and Bad News on Twitter

… We use tf-idf representation for each vocabulary term. 5.2.3 Embeddings Finally, we also use fasttext based embedding (Mikolov et al., 2018) vectors which are trained on common crawl having 600 billion tokens. 5.3 Classifiers …


Identifying and Addressing Structural Inequalities in the Representativeness of Geographic Technologies

… knowledge graphs (Wikipedia and Google [289]), word embeddings (Wikipedia, Twitter, and Common Crawl in GloVe embeddings [238]), object detection (Instagram hashtags and Facebook [292])—and adding …


P Watson, B Bhattacharjee, NC CODELLA… - US Patent App. 15/982,622, 2019

US20190354850A1 - Identifying transfer models for machine learning tasks - Google Patents. Identifying transfer models for machine learning tasks. Download PDF Info. Publication number US20190354850A1. US20190354850A1 …


Idiap Abstract Text Summarization System for German Text Summarization Task

… The experiments performed over 1http://opennmt.net/OpenNMT-py/ Summarization.html 2https://www.swisstext.org/ 3http://commoncrawl.org/ these datasets are described in the Section 4 (de- noted as S1 experimental …


IIT Varanasi at HASOC 2019: Hate Speech and Offensive Content Identification in Indo-European Languages

A Mishra, S Pal - Proceedings of the 11th annual meeting of the Forum …

… embedding. One of the pretrained glove embeddings is based on the common crawl which represents each word in the dimension of 300, and the other one is based on Twitter data which represents each word in the dimension of 200 …


IIT-BHU at CIQ 2019: Classification of Insincere Questions

… Different versions of glove pre-trained em- bedding exist; however, we use embedding trained of dimension 300 on common crawl using 840B tokens and 2.2M vocabulary3. We generated random embedding of dimension 300 for out of vocabulary words …


Impact of Debiasing Word Embeddings on Information Retrieval

… Bolukbasi et al. [1] show that there is a high correlation in bias in Word2Vec trained on Google News and Glove trained on the common crawl, so we still cannot infer whether the method or the dataset is more important for creating the bias …


Improved Quality Estimation of Machine Translation with Pre-trained Language Representation

G Miao, H Di, J Xu, Z Yang, Y Chen, K Ouchi - CCF International Conference on …, 2019

… The former is mainly obtained from the open news datasets of the WMT17 and WMT18 MT evaluation tasks, including five data sets: Europarl v7, Europarl v12, Europarl v13, Common Crawl corpus, and Rapid corpus of EU press releases …


Improving Conditioning in Context-Aware Sequence to Sequence Models

X Wang, J Weston, M Auli, Y Jernite - arXiv preprint arXiv:1911.09728, 2019

… 2019) for LFQA. The dataset consists of 272,000 complex questions and answer pairs, along with supporting documents created by gathering and concatenating passages from CommonCrawl web pages which are relevant to the question …


Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Data

W Zhao, L Wang, K Shen, R Jia, J Liu - arXiv preprint arXiv:1903.00138, 2019

… We do not use reranking when evaluating the CoNLL-2014 data sets. But we rerank the top 12 hypothesizes us- ing the language model trained on Common Crawl (Junczys-Dowmunt and Grundkiewicz, 2016) for …


Improving Implicit Stance Classification in Tweets Using Word and Sentence Embeddings

R Schaefer, M Stede - Joint German/Austrian Conference on Artificial …, 2019

… combinations. 4.2 fastText Embeddings. We use pre-trained 300-dimensional fastText [11] word vectors that have been trained on Wikipedia and Common Crawl data. For training, an extension of the CBOW model has been used …


Improving Named Entity Recognition with Commonsense Knowledge Pre-training

G Dekhili, NT Le, F Sadat - Pacific Rim Knowledge Acquisition Workshop, 2019

… which is the concatenation of ConceptNet PPMI embeddings with Word2Vec embeddings trained on 100 billion words of Google News using skip-grams with negative sampling [14] and GloVe 1.2 embeddings trained on 840 billion words of the Common Crawl [16] …


Improving Neural Machine Translation of Subtitles with Finetun-ing

… 3 Results 53 3.1 ParallelCorpora . . . . . 54 3.1.1 Europarl. . . . . 54 3.1.2 Common Crawl . . . . . 54 3.1.3 NewsCommentary . . . . . 56 3.1.4 Subtitles …


Improving Neural Machine Translation Robustness via Data Augmentation: Beyond Back Translation

Z Li, L Specia - arXiv preprint arXiv:1910.03009, 2019

… 3.1 Corpora We used all parallel corpora from the WMT19 Robustness Task on Fr↔En. For out-of-domain training, we used the WMT15 Fr↔En News Translation Task data, including Europarl v7, Common Crawl, UN, News Commentary v10, and Gigaword Corpora …


Improving Neural Machine Translation with Pre-trained Representation

R Weng, H Yu, S Huang, W Luo, J Chen - arXiv preprint arXiv:1908.07688, 2019

… We use newstest2015 (NST15) as our validation set, and newstest2016 (NST16) as test sets 4. We use 40 million monolingual sentences from WMT-16 Common Crawl data-set … We use 5 million monolingual sentences …


Improving orienteering-based tourist trip planning with social sensing

F Persia, G Pilato, M Ge, P Bolzoni, D D'Auria… - Future Generation …, 2019

… This is a popular technique in machine learning for uncovering subsymbolic meanings, such as word analogies. We utilized a pre-trained word vector encoding for Italian provided by fastText [32], which was trained on Common Crawl and Wikipedia …


Improving Quality Estimation of Machine Translation by Using Pre-trained Language Representation

G Miao, H Di, J Xu, Z Yang, Y Chen, K Ouchi - China Conference on Machine …, 2019

Y Chen, K Ouchi - Machine Translation: 15th China Conference, CCMT …, 2019

… Metrics We first train the bilingual expert model [9] with large-scale parallel corpus released for the WMT17/WMT18 News Machine Translation Task, which mainly consists of five data sets, including Europarl v7, Europarl v12 …


Improving Question Answering over Incomplete KBs with Knowledge-Aware Reader

W Xiong, M Yu, S Chang, X Guo, WY Wang - arXiv preprint arXiv:1905.07098, 2019

… Page 6. A Implementation Details Throughout our experiments, we use the 300-dimension GloVe embeddings trained on the Common Crawl corpus. The hidden dimension of LSTM and the dimension of entity embeddings are both 100 …


In-call virtual assistant

R Raanani, R Levy, MY Breakstone - US Patent App. 16/165,566, 2019

… At the same time, natural language processing (NLP) approaches to both topic modeling and world-knowledge modeling, have become much more efficient due to the availability of large, freely accessible natural language corpora (eg, CommonCrawl), as well as freely …


Incendiary News Detection

… features. We run classification experiments for unigrams, and combination of uniand bi-grams. 10https://www.nltk.org/ 11http://scikit-learn.org 12https://github.com/ otuncelli/turkish-stemmer-python 13http://commoncrawl.org/ For …


Incorporating Domain Knowledge into Natural Language Inference on Clinical Texts

M Lu, Y Fang, F Yan, M Li - IEEE Access, 2019

… two domain-specific corpus: • GloVe[CC]: GloVe embeddings [21], trained on Common Crawl. • fastText[BioASQ]: fastText embeddings [22], trained on PubMed abstracts from the BioASQ challenge [23]. • fastText[MIMIC-III]: fastText …


Incorporating Syntactic Knowledge in Neural Quality Estimation for Machine Translation

N Ye, Y Wang, D Cai - China Conference on Machine Translation, 2019

… One is the large-scale bilingual dataset for training the feature extraction module. It comes from the parallel corpus of WMT machine translation task, including Europarl v7, Common Crawl corpus, News Commentary v11 and so on …


Inducing Relational Knowledge from BERT

Z Bouraoui, J Camacho-Collados, S Schockaert - arXiv preprint arXiv:1911.12753, 2019

… As static word embeddings for the baselines, we will use the Skip-gram word vectors that were pre-trained from the 100B words Google News data set6 (SG-GN) and GloVe word vectors which were pre-trained from the …


Inducing Schema. org Markup from Natural Language Context

GK Shahi, D Nandini, S Kumari - Kalpa Publications in Computing, 2019

… extension, in 2012 another data hub called Web Data Commons [5] came up with structured data extracted from the Common Crawl … 5http:// commoncrawl.org/ 6http://webdatacommons.org/ 7The WARC file format …


M Le, S Roller, L Papaxanthos, D Kiela, M Nickel - arXiv preprint arXiv:1902.00913, 2019

Page 1. Inferring Concept Hierarchies from Text Corpora via Hyperbolic Embeddings Matt Le1 and Stephen Roller1 and Laetitia Papaxanthos2 Douwe Kiela1 and Maximilian Nickel1 1Facebook AI Research, New York …


Information extraction

… 8 Page 9. Taxi [Panchenko et al., 2016] 1. Crawl domain-specific text corpora in addition to WP, Commoncrawl 2. Candidate hypernymy extraction 1. Via substrings • “biomedical science” isA “science” • “microbiology” isA “biology” • “toast with bacon” isA “toast” …


InriaFBK Drawing Attention to Offensive Language at Germeval2019

M Corazza, S Menini, E Cabrio, S Tonelli, S Villata…

… This is the main reason why we chose to use FastText embeddings (Bojanowski et al., 2016), pretrained on Common Crawl and Wikipedia 3. 4.3 Recurrent model We develop a simple recurrent neural network model and use it for all subtasks …


Integrating Grammatical Features into CNN Model for Emotion Classification

AC Le - 2018 5th NAFOSTED Conference on Information and …, 2018

… a sentence s = 11 In this study we used the vector set GloVe [16], it is pretrained word vectors for Common Crawl (glove.42B.300d) with 300 dimensions for word embeddings to use for English data. For Vietnamese emotion …


Integrating UMLS for Early Detection of Sings of Anorexia

FM Plaza-del-Arco, P López-Úbeda, MC Dıaz-Galiano… - 2019

… Specifically, we use Page 6. the available pre-trained statistical models for English ”en core web md” wich version is 1.2.0. It is composed of 685k keys, 20k unique vectors (300 dimensions) and it was trained on OntoNotes …


Integrating word embeddings and document topics with deep learning in a video classification framework

Z Kastrati, AS Imran, A Kurti - Pattern Recognition Letters, 2019

… GloVe contains word embeddings for a vocabulary of 400K words trained on 42 billion words from Wikipedia pages and newswire, and fastText includes word embeddings for a vocabulary of 2 million words trained on 600 billion tokens from Common Crawl …


Intelligent sentiment analysis approach using edge computing‐based deep learning technique

H Sankar, V Subramaniyaswamy, V Vijayakumar… - Software: Practice and Experience

… Word2Vec, 300d, 3 Million, 100 Billion. Common Crawl, 300d, 42 Billion, 1.9 Million. Common Crawl, 300d, 840 Billion, 2.2 Million. The main drawback of unsupervised word embedding learning is that it does not hold the sentiment …


X Yuan, MA Cote, J Fu, Z Lin, C Pal, Y Bengio… - arXiv preprint arXiv …, 2019

Page 1. Interactive Language Learning by Question Answering Xingdi Yuan♥∗ Marc-Alexandre Côté♥∗ Jie Fu♣♠ Zhouhan Lin♦♠ Christopher Pal♣♠ Yoshua Bengio♦♠ Adam Trischler♥ ♥Microsoft Research, Montréal ♣Polytechnique …


Interactive Machine Comprehension with Information Seeking Agents

X Yuan, J Fu, MA Cote, Y Tay, C Pal, A Trischler - arXiv preprint arXiv:1908.10449, 2019

… Word embeddings are initialized by the 300-dimension fastText (Mikolov et al. 2018) vectors trained on Common Crawl (600B tokens), and are fixed during training. Character embeddings are initialized by 200-dimension random vectors …


Internet of Things Anomaly Detection using Multivariate Analysis

S Ezekiel, AA Alshehri, L Pearlstein, XW Wu, A Lutz - The 3rd ICICPE 2019 Conference …

… Our model uses the GloVe (Pennington et al., 2014) 300-dimensional vectors trained on the Common Crawl corpus with 42B tokens as word level features, as this resulted in the best performance in preliminary experiments …


Iot-based call assistant device

R Raanani, R Levy, MY Breakstone - US Patent App. 16/168,663, 2019


Iterative Keyword Optimization

… The model was trained on Common Crawl 4 and Wikipedia 5 using the fastText library 6. We used Euclidean as the distance measure … 4 http://commoncrawl.org/ 5 https://www.wikipedia.org/ 6 https://fasttext …


JHU 2019 Robustness Task System Description

M Post, K Duh - Proceedings of the Fourth Conference on Machine …, 2019

… the best million lines each of CommonCrawl, Gigaword, and the UN corpus; and • the MTNT training data. Data sizes are indicated in Table 1. dataset segments words Europarl 2.0m 50.2m News Commentary 200k 4.4m …


Johns Hopkins University Submission for WMT News Translation Task

K Marchisio, YK Lal, P Koehn - Proceedings of the Fourth Conference on Machine …, 2019

… sampled bitext (x2). ParaCrawl1 and Common Crawl2 are filtered similarly, and added to form the training set for the final models. We … Crawl. ParaCrawl and Common Crawl were combined into a single corpus before filtering …


Joint Training for Neural Machine Translation

Page 1. Springer Recognizing Theses Outstanding Ph.D. Research Yong Cheng Joint Neural Translation Training Machine for Page 2. Springer Theses Recognizing Outstanding Ph.D. Research Page 3. Aims and Scope The …


Jointly Learning to Align and Translate with Transformer Models

S Garg, S Peitz, U Nallasamy, M Paulik - arXiv preprint arXiv:1909.02074, 2019

… by Vilar et al. (2006). We use all available bilingual data (Europarl v7, Common Crawl corpus, News Commentary v13 and Rapid corpus of EU press releases) excluding the ParalCrawl corpus. We remove sentences longer …


JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus

M Morishita, J Suzuki, M Nagata - arXiv preprint arXiv:1911.10668, 2019

… To select the candidate domains, we first identified the language of all the Common Crawl text data by CLD26 and counted how much … Since the crawled data stored on Common Crawl may not contain the entire website or might …


KaWAT: A Word Analogy Task Dataset for Indonesian

K Kurniawan - arXiv preprint arXiv:1906.09912, 2019

… We used fastText pretrained embeddings introduced in (Bojanowskietal.,2017) and (Grave et al., 2018), which have been trained on Indonesian Wikipedia and Indonesian Wikipedia plus Common Crawl data respectively. We …


Keyphrase Extraction from Scholarly Articles as Sequence Labeling using Contextualized Embeddings

D Sahrawat, D Mahata, M Kulkarni, H Zhang… - arXiv preprint arXiv …, 2019

… and OpenAI GPT-2 (small, medium). As a baseline, we also use 300 dimensional fixed embeddings from Glove2, Word2Vec3, and FastText4 (common-crawl, wiki-news). We also compare the proposed architecture against …


KiloGrams: Very Large N-Grams for Malware Classification

E Raff, W Fleming, R Zak, H Anderson, B Finlayson… - arXiv preprint arXiv …, 2019

… A ccuracy s = 1 s = ⌈n/4⌉ Figure 1: Balanced Accuracy results (y-axis) on the Public PDF dataset as we increase then-gram size (x-axis, log-scale), and alter the hashing stride s. Using a hashing-stride retains more …


KIT's Submission to the IWSLT 2019 Shared Task on Text Translation

… We made use of all allowed data, which is broken down in table 1. The allowed parallel data from WMT consists of Commoncrawl, CzEng (which makes up the vast majority of the parallel training data), Europarl, news commentrary and paracrawl …


Knowledge empowered prominent aspect extraction from product reviews

Z Luo, S Huang, KQ Zhu - Information Processing & Management, 2019


Knowledge Graph-Driven Conversational Agents

… We use a CNN with max pooling and pretrained Glove embeddings trained on the Common Crawl 840B dataset [6] [7]. By applying our CNN classifier as a straightforward 1-of-k document classification task, we are able to achieve …


Knowledge-based Conversational Search

S Vakulenko - arXiv preprint arXiv:1912.06859, 2019

Page 1. arXiv:1912.06859v1 [cs.IR] 14 Dec 2019 Page 2. Page 3. Knowledge-based Conversational Search DISSERTATION submitted in partial fulfillment of the requirements for the degree of Doktorin der Technischen Wissenschaften by Svitlana Vakulenko, MSc …


Kyoto University participation to the WMT 2019 news shared task

F Cromieres, S Kurohashi - Proceedings of the Fourth Conference on Machine …, 2019

… Page 2. 164 3 Data preprocessing 3.1 Data used For bilingual data, we used the provided corpora: europarl (≈ 1.7M sentence pairs), common crawl(≈ 620k sentence pairs) and newscommentary (≈ 255k sentence pairs). We did not use the paracrawl corpus …


Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation

D Loureiro, A Jorge - arXiv preprint arXiv:1906.10007, 2019

… tokens in the sentence. We choose fastText (Bojanowski et al., 2017) embeddings (pretrained on CommonCrawl), which are biased towards morphology, and avoid Out-of-Vocabulary issues as explained in §2.1. We use fastText …


Language Models are Unsupervised Multitask Learners

A Radford, J Wu, R Child, D Luan, D Amodei…

… A promising source of diverse and nearly unlimited text is web scrapes such as Common Crawl … Trinh & Le (2018) used Common Crawl in their work on commonsense reasoning but noted a large amount of documents “whose content are mostly unintelligible” …


Language Models with Pre-Trained (GloVe) Word Embeddings

… Despite the huge size of the Common Crawl corpus, some words may not exist with the embeddings, so we set these words to random vectors, and use the same embeddings consistently if we encounter the same unseen word again in the text …


Large Memory Layers with Product Keys

G Lample, A Sablayrolles, MA Ranzato, L Denoyer… - arXiv preprint arXiv …, 2019

… Experiments Page 20. Dataset 20 ▶ Extracted from the public Common Crawl. ▶ 40 million English news articles in training set, 5000 in validation and test set each. ▶ Did not shuffle sentences, allowing the model to learn …


Large Scale Linguistic Processing of Tweets to Understand Social Interactions among Speakers of Less Resourced Languages: The Basque Case

J Fernandez de Landa, R Agerri, I Alegria - Information, 2019

… resourced languages such as Basque. However, FastText provides pre-trained models for many languages, including Basque [33] by using the common crawl data (http://commoncrawl.org). The Basque model they distribute …


Page 1. Last-Mile TLS Interception: Analysis and Observation of the Non-Public HTTPS Ecosystem Xavier de Carné de Carnavalet A thesis in The Concordia Institute for Information Systems Engineering Presented …


Latent Question Interpretation Through Parameter Adaptation

T Parshakova, F Rameau, A Serdega, I Kweon, DS Kim - IEEE/ACM Transactions on …, 2019

… A. Implementation Details For the sake of reproducibility, we provide the technical details related to the implementation of our approach. First of all, the initial word embeddings are initialized with GloVe embeddings, which …


Laying the foundations for benchmarking open data automatically: a method for surveying data portals from the whole web

A Sheffer Correa, F Soares Correa Da Silva - 20th Annual International Conference …, 2019

… KEYWORDS Open Data, Common Crawl, CKAN, Socrata, ArcGIS, OpenDataSoft … Common Crawl conducts crawls once a month and persists all the content in Web Archive (WARC) file format to allow multibillion web page archives with hundreds of terabytes in size …


LCEval: Learned Composite Metric for Caption Evaluation

N Sharif, L White, M Bennamoun, W Liu, SAA Shah

… Table 1: The details of pre-trained embeddings used in our experiments Name Source Dimensions Corpus Corpus Size Vocabulary Size GloVE 840B 300d [40] 300 Common Crawl 8.40E+11 2.20E+06 Word2vec Google 300d [34] …


Learning as the Unsupervised Alignment of Conceptual Systems

BD Roads, BC Love - arXiv preprint arXiv:1906.09012, 2019

… We found that alignment correlations positively correlated with mapping accuracy across a variety of scenarios (Figure 3A-C). The three conceptual systems were derived from a Common Crawl text corpus (Pennington et …


Learning from Personal Longitudinal Dialog Data

C Welch, V Pérez-Rosas, JK Kummerfeld, R Mihalcea…

… Message Embeddings: We also obtain word vector representations for each message using the GloVe Common Crawl pre-trained model.19 We chose this word embedding over other off-theshelf options because the Common …


Learning multilingual topics through aspect extraction from monolingual texts

J Huber, M Spiliopoulou - Proceedings of the Fifth International Workshop on …, 2019

… Xu et al., 2018). It was trained on the CommonCrawl corpus, a general-purpose text corpus that includes text from several billion web pages; the GloVe embeddings were trained on 840 billion tokens. The GloVe set includes …


Learning Outside the Box: Discourse-level Features Improve Metaphor Identification

J Mu, H Yannakoudakis, E Shutova - arXiv preprint arXiv:1904.02246, 2019

… To learn representations, we use several widelyused embedding methods:4 GloVe We use 300-dimensional pre-trained GloVe embeddings (Pennington et al., 2014) trained on the Common Crawl corpus as representations of a lemma and its arguments …


Learning Relational Fractals for Deep Knowledge Graph Embedding in Online Social Networks

J Zhang, L Tan, X Tao, D Wang, JJC Ying, X Wang - International Conference on Web …, 2019

… Our twitter dataset was live streamed from a twitter API account and contains a maximum of 1675882 nodes and 160799842 links. The Google dataset was obtained from the repositories of common crawl and was sentilyzed from the stripped down WET file contents …


Learning to Generate Personalized Product Descriptions

G Elad, I Guy, K Radinsky, S Novgorodov, B Kimelfeld - 2019

… For the title representation, we used fastText word embeddings2 pre-trained on Common Crawl and Wikipedia [25, 33], weighted based on each word's TF-IDF score [4].3 In addition, we included as features the participant's demo …


J Urbanek, A Fan, S Karamcheti, S Jain, S Humeau… - arXiv preprint arXiv …, 2019

Page 1. Learning to Speak and Act in a Fantasy Text Adventure Game Jack Urbanek1 Angela Fan1,2 Siddharth Karamcheti1 Saachi Jain1 Samuel Humeau1 Emily Dinan1 Tim Rocktäschel1,3 Douwe Kiela1 Arthur Szlam1 Jason …


Learning Word Ratings for Empathy and Distress from Document-Level User Responses

J Sedoc, S Buechel, Y Nachmany, A Buffone, L Ungar - arXiv preprint arXiv …, 2019

… (2013) using 10-fold crossvalidation. For word embeddings we used off-the-shelf Fasttext subword embeddings (Mikolov et al., 2018).4 The embeddings are trained with subword information on Common Crawl (600B tokens) …


Leveraging Distributional and Relational Semantics for Knowledge Extraction from Textual Corpora

G ROSSIELLO, G SEMERARO, M DI CIANO - 2019


Leveraging End-to-End Speech Recognition with Neural Architecture Search

A Baruwa, M Abisiga, I Gbadegesin, A Fakunle - arXiv preprint arXiv:1912.05946, 2019

… We train a 3-gram, 5-gram and a 7-gram language model on common crawl 1. The relative performances are summarised in tables 1 and 2. Decoding is done by beam-searching for the output y that maximizes φ(c) given by …


O Feyisetan, T Diethe, T Drake - arXiv preprint arXiv:1910.08917, 2019

Page 1. Leveraging Hierarchical Representations for Preserving Privacy and Utility in Text Oluwaseyi Feyisetan Amazon sey@amazon.com Tom Diethe Amazon tdiethe@amazon.co.uk Thomas Drake Amazon draket@amazon.com …


Leveraging Pretrained Image Classifiers for Language-Based Segmentation

D Golub, R Martín-Martín, A El-Kishky, S Savarese - arXiv preprint arXiv:1911.00830, 2019

… With Word2Vec we first embed the target labels l and the labels in the set of possible proxy labels in a shared vector space using 300-dimensional GloVe embeddings [29] trained on the Common Crawl 840B word corpus. For labels that contains multiple words …


Leveraging Unpaired Out-of-Domain Data for Image Captioning

X Chen, M Zhang, Z Wang, L Zuo, B Li, Y Yang - Pattern Recognition Letters, 2018


Leveraging Web Semantic Knowledge in Word Representation Learning

… We extract a large collection of semantic lists from the Common Crawl data7 using the patterns defined in Table 1 and filter out entries that do not exist in the vocabulary of the training data … 6http://dumps.wikimedia.org/enwiki/ 7http://commoncrawl.org/ Page 5 …


Limsi-multisem at the ijcai semdeep-5 wic challenge: Context representations for word usage similarity estimation

AG Soler, M Apidianaki, A Allauzen - Proceedings of the 5th Workshop on Semantic …, 2019

… Di- mensionality reduction is applied to a weighted average of the vectors of words in a sentence. Weighting is based on word frequency in Common Crawl. We use SIF in combination with 300- d GloVe vectors trained …


Lingua Custodia at WMT'19: Attempts to Control Terminology

F Burlot - arXiv preprint arXiv:1907.04618, 2019

… to the decoder. Page 2. 2 Baseline The training parallel data provided for the task consisted of nearly 10M sentences, including Europarl (Koehn, 2005), Common-crawl, Newscommentary and Bicleaner07. The former was …


Linked Open Data Validity--A Technical Report from ISWS 2018

TA Ghor, E Agrawal, M Alam, O Alqawasmeh… - arXiv preprint arXiv …, 2019

Page 1. Linked Open Data Validity A Technical Report from ISWS 2018 April 1, 2019 Bertinoro, Italy arXiv:1903.12554v1 [cs.DB] 26 Mar 2019 Page 2. Authors Main Editors Mehwish Alam, Semantic Technology Lab, ISTC-CNR …


Linking artificial and human neural representations of language

J Gauthier, R Levy - arXiv preprint arXiv:1910.01244, 2019

… contrasts between the 384 sentences tested. 9We use publicly available GloVe vectors computed on Common Crawl, available in the spaCy toolkit as en vectors web lg. Page 6. 3 Results We first present the performance of …


LINSPECTOR: Multilingual Probing Tasks for Word Representations

GG Şahin, C Vania, I Kuznetsov, I Gurevych - arXiv preprint arXiv:1903.09442, 2019

Page 1. LINSPECTOR Multilingual Probing Tasks for Word Representations Gözde Gül Sahin∗ UKP Lab / TU Darmstadt Clara Vania∗∗ ILCC / University of Edinburgh Ilia Kuznetsov UKP Lab / TU Darmstadt Iryna Gurevych UKP Lab / TU Darmstadt …


LIUM's Contributions to the WMT2019 News Translation Task: Data and Systems for German-French Language Pairs

F Bougares, J Wottawa, A Baillot, L Barrault, A Bardet - … 2: Shared Task Papers, Day 1 …, 2019

… As it can be seen from tables 1 and 2, the effect of the cleaning step is more pronounced for the noisy parallel corpora (ie ParaCrawl and Common Crawl) … Page 3. 131 #lines #token FR #token DE europarl-v7 1.7M 45.9M 40.9 …


Local bow-tie structure of the web

Y Fujita, Y Kichikawa, Y Fujiwara, W Souma, H Iyetomi - Applied Network Science, 2019

… This fact means that the absence of self-similarity between page level and host/domain levels. Meusel et al. (2014, 2015) investigated the publicly accessible crawl of the web gathered by the Common Crawl Foundation in 2012 (CC12) (Meusel et al. 2014; 2015) …


Logical Layout Analysis using Deep Learning

… of the text zones. GloVE provides 300 dimensional vectors, one vector for each word. We have used the one trained on common crawl having 840 billion tokens and vectors for a total of 2.2 million words. Since we also want …


Longitudinal Analysis of Misuse of Bitcoin⋆

… its labels). Seed data was used from previously published onion data sets, references to onions in a large collection of DNS resolver logs, and an open repository of (non-onion) web crawl data, called the Common Crawl. The …


Look Who's Talking: Inferring Speaker Attributes from Personal Longitudinal Dialog

C Welch, V Pérez-Rosas, JK Kummerfeld, R Mihalcea - arXiv preprint arXiv …, 2019

… The word embedding inputs to the context encoder are 300 dimensional. 8 Features Word Embeddings: We obtain word vector representations for each message using the GloVe Common Crawl pre-trained model [12]. We …


Low Resource Sequence Tagging with Weak Labels

… For FAMULUS, we use 300-dimensional German fastText embeddings (Grave et al. 2018), and for NER and PICO we use 300-dimensional English GloVe 3 embeddings trained on 840 billion tokens from Common Crawl. To …


Low Supervision, Low Corpus size, Low Similarity! Challenges in cross-lingual alignment of word embeddings: An exploration of the limitations of cross-lingual word …

Page 1. Low Supervision, Low Corpus size, Low Similarity! Challenges in cross-lingual alignment of word embeddings An exploration of the limitations of cross-lingual word embedding alignment in truly low resource scenarios Andrew Dyer …


LSTM for Dialogue Breakdown Detection: Exploration of Different Model Types and Word Embeddings

… The words are uncased. GloVe Common Crawl … The results presented in the Table 2, allow to conclude that GloVe Common Crawl demonstrate the best performance, the GloVe Twitter being the second best, the word2vec Google News is the worst. Page 9 …


LTL-UDE at SemEval-2019 Task 6: BERT and Two-Vote Classification for Categorizing Offensiveness

P Aggarwal, T Horsmann, M Wojatzki, T Zesch - … of the 13th International Workshop on …, 2019

… word representations. The resulting posting vector is re-scaled into the range zero to one. We use the pre-trained embeddings provided by Mikolov et al. (2018), which are trained on the common crawl corpus. Classifiers We …


ltl. uni-due at SemEval-2019 Task 5: Simple but Effective Lexico-Semantic Features for Detecting Hate Speech in Twitter

H Zhang, M Wojatzki, T Horsmann, T Zesch - … of the 13th International Workshop on …, 2019

… of LSTMs and CNNs (LSTM + CNN). We initialize all setups with the 300-dimensional word embeddings provided by Mikolov et al. (2018), which were trained on the common crawl corpus. Furthermore, in all setups, we use …


Page 1. Master degree course in Computer Engineering Master Degree Thesis Machine Reading of Clinical Notes for Automated ICD Coding Supervisor Prof. Maurizio Morisio Candidate Stefano Malacrin`o Internship tutors …


Machine Translation of Restaurant Reviews: New Corpus for Domain Adaptation and Robustness

A Bérard, I Calapodescu, M Dymetman, C Roux… - arXiv preprint arXiv …, 2019

… data, we built a new training corpus named UGC (User Generated Content), closer to our domain, by combining: Multi UN, OpenSubtitles, Wikipedia, Books, Tatoeba, TED talks, ParaCrawl11 and Gourmet12 (See Table 3) …


Mapping languages and demographics with georeferenced corpora

… To answer this question, we collect and analyze two large global-scale datasets: web-crawled data from the Common Crawl (16.65 billion words) and social media data from Twitter (4.14 billion words). This paper evaluates demographic-type informa …


Massive vs. Curated Word Embeddings for Low-Resourced Languages. The Case of Yor\ub\'a and Twi

JO Alabi, K Amponsah-Kaakyire, DI Adelani… - arXiv preprint arXiv …, 2019

… The resource par excellence is Wikipedia2, an online encyclopedia currently available in 307 languages3. Other initiatives such as Common Crawl4 or the Jehovahs Witnesses site5 are also repositories for multilingual …


Massively multilingual transfer for NER

A Rahimi, Y Li, T Cohn - Proceedings of the 57th Conference of the Association …, 2019

Page 1. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 151–164 Florence, Italy, July 28 - August 2, 2019. c 2019 Association for Computational Linguistics 151 Massively Multilingual Transfer for NER …


MASTER UNIVERSITARIO EN INGENIERÍA DE TELECOMUNICACION

Page 1. M´ASTER UNIVERSITARIO EN INGENIERÍA DE TELECOMUNICACI´ON TRABAJO FIN DE M´ASTER DESING AND DEVELOPMENT OF A HATE SPEECH DETECTOR IN SOCIAL NETWORKS BASED ON DEEP LEARNING TECHNOLOGIES …


Measuring Gender Bias in Word Embeddings across Domains and Discovering New Gender Bias Word Categories

K Chaloner, A Maldonado - Proceedings of the First Workshop on Gender Bias in …, 2019

… WEAT's authors applied these tests to the publicly-available GloVe embeddings trained on the English-language “Common Crawl” corpus (Pennington et al., 2014) as well as the Skip-Gram (word2vec) embeddings …


Medical Word Embeddings for Spanish: Development and Evaluation

F Soares, M Villegas, A Gonzalez-Agirre, M Krallinger… - Proceedings of the 2nd …, 2019

… makes available Word2Vec models pre-trained on about 100 billion words from Google News corpus in English1. Regarding other languages, on FastText website2 one can download pre-trained embeddings for 157 lan …


Meemi: Finding the Middle Ground in Cross-lingual Word Embeddings

Y Doval, J Camacho-Collados, L Espinosa-Anke… - arXiv preprint arXiv …, 2019

… 10 Page 11. the WaCky project [23], containing 2 and 0.8 billion words, respectively.6 For Finnish and Russian, we use their corresponding Common Crawl monolingual corpora from the Machine Translation of News Shared Task 20167, composed of …


Membership Inference Attacks on Sequence-to-Sequence Models

S Hisamoto, M Post, K Duh - arXiv preprint arXiv:1904.05506, 2019

… For example, e (d) i with d = l1 and i = 1 might refer to the first sentence in the Europarl subcorpus, while e (d) i with d = l2 and i = 1 might refer to the first sentence in the CommonCrawl subcorpus … CommonCrawl 5,000 5,000 2,389,123 2,379,123 N/A …


Metaphor Interpretation Using Word Embeddings

… relatively large corpus. Specifically, we use DepCC,1 a dependency-parsed “web-scale corpus” based on CommonCrawl.2 There are 365 million documents in the corpus, comprising about 252B tokens. Among other preprocessing …


Methods and apparatus for detection of malicious documents using machine learning

JD Saxe, R HARANG - US Patent App. 16/257,749, 2019

… decision tree, etc.). The memory 120 includes one or more datasets 112 (eg, a VirusTotal dataset and/or a Common Crawl dataset, as described in further detail below) and one or more training models 124. The malware detection …


Microsoft Research Asia's Systems for WMT19

Y Xia, X Tan, F Tian, F Gao, W Chen, Y Fan, L Gong…

… Dataset We concatenate “Europarl v9”, “News Commentary v14”, “Common Crawl corpus” and “Document-split Rapid corpus” as the ba- sic bilingual … We merge the “commoncrawl”, “europarl-v7” and part of “de-fr.bicleaner07” …


MIDAS: A Dialog Act Annotation Scheme for Open Domain Human Machine Spoken Conversations

D Yu, Z Yu - arXiv preprint arXiv:1908.10023, 2019

… An example can be seen in the last USER2 utterance in Table 1. Word em- beddings are pre-trained with fastText (Mikolov et al., 2018) using Common Crawl. We evaluate the segmentation model on human labeled 2K human utterances of collected data …


Mining Discourse Markers for Unsupervised Sentence Representation Learning

D Sileo, T Van-De-Cruys, C Pradel, P Muller - arXiv preprint arXiv:1903.11850, 2019

… We use sentences from the Depcc corpus (Panchenko et al., 2017), which consists of En- glish texts harvested from commoncrawl web data … Word embeddings are fixed GloVe embeddings with 300 dimensions, trained …


Mix-review: Alleviate Forgetting in the Pretrain-Finetune Framework for Neural Language Generation Models

T He, J Liu, K Cho, M Ott, B Liu, J Glass, F Peng - arXiv preprint arXiv:1910.07117, 2019

… For pre-training, we use the large-scale CCNEWS data (Bakhtin et al., 2019) which is a de- duplicated subset of the English portion of the CommonCrawl news data-set1. The dataset contains news articles published worldwide …


MLT-DFKI at CLEF eHealth 2019: Multi-label Classification of ICD-10 Codes with BERT

S Amin, G Neumann, K Dunfield, A Vechkaeva… - CLEF (Working Notes), 2019

… have stronger linguistic signals to classify the classes where German models make mistakes [1]. The baseline proved to be a strong one, with the highest precision of all and outperforming HAN and CNN models, for both German …


Mono-and Cross-lingual Semantic Word Similarity for Urdu Language

Page 1. I Monoand Cross-lingual Semantic Word Similarity for Urdu Language By Ghazeefa Fatima CIIT/FA17-RCS-016/LHR MS Thesis In Computer Science COMSATS University Islamabad Lahore Campus Page …


MoRTy: Unsupervised Learning of Task-specialized Word Embeddings by Autoencoding

N Rethmeier, B Plank - Proceedings of the 4th Workshop on Representation …, 2019

… Hence, we demonstrate the method's application for single-task, multi-task, small, medium and web-scale (common crawl) corpus-size settings (Section 4). Learning to scale-up by pretraining on more (un-)labeled data is both: (a) not always possible in low-resource …


Multi-class Document Classification Using Improved Word Embeddings

BA Rabut, AC Fajardo, RP Medina - Proceedings of the 2nd International Conference …, 2019

… ACM ISBN 978-1-4503-7290-9/19/10…$15.00 https://doi.org/10.1145/3366650.3366661 42 Page 2. Common crawl)[7]. The pre-trained word embedding vectors serve as input in the classification algorithm for evaluation and prediction …


Multi-domain Dialogue State Tracking as Dynamic Knowledge Graph Enhanced Question Answering

L Zhou, K Small - arXiv preprint arXiv:1911.06192, 2019

… For experiments with GloVe embeddings, we use GloVe embeddings pre-trained on Common Crawl dataset.3 The dimension of GloVe embeddings is 300, and the dimension of character-level embeddings is 100, such that Dw = 400 …


Multi-Granular Text Encoding for Self-Explaining Categorization

Z Wang, Y Zhang, M Yu, W Zhang, L Pan, L Song, K Xu… - arXiv preprint arXiv …, 2019

… for each set. Hyperparameters We use the 300-dimensional GloVe word vectors pre-trained from the 840B Common Crawl corpus (Pennington et al., 2014), and set the hidden size as 100 for node embeddings. We apply dropout …


Y Feldman, R El-Yaniv - arXiv preprint arXiv:1906.06606, 2019

Page 1. Multi-Hop Paragraph Retrieval for Open-Domain Question Answering Yair Feldman and Ran El-Yaniv Department of Computer Science Technion – Israel Institute of Technology Haifa, Israel {yairf11, rani}@cs.technion.ac.il Abstract …


Multi-Resolution Models for Learning Multilevel Abstract Representation with Application to Information Retrieval

Page 1. MULTI-RESOLUTION MODELS FOR LEARNING MULTILEVEL ABSTRACT REPRESENTATION WITH APPLICATION TO INFORMATION RETRIEVAL A Dissertation Submitted to the Graduate School University of Arkansas at Little Rock …


Multi-Team: A Multi-attention, Multi-decoder Approach to Morphological Analysis.

A Ustün, R van der Goot, G Bouma, G van Noord

… 2018). For FastText, two sets of pre-trained embeddings are available: one is trained only on Wikipedia (Bojanowski et al., 2017), whereas the newer versions are also trained on CommonCrawl (Grave et al., 2018). Whenever …


Multilingual Culture-Independent Word Analogy Datasets

M Ulčar, M Robnik-Šikonja - arXiv preprint arXiv:1911.10038, 2019

… language is shown in the Table 6. Table 6: Percentage of constructed analogy pairs covered by the first 200,000 word vectors from common crawl fastText embeddings. Language Coverage (%) Croatian 81.67 English 97.05 …


Multilingual Fake News Detection with Satire

G Guibon, L Ermakova, H Seffih, A Firsov…

… Detection of Deception. Non-verbal communication (2014), https://nvc.uvt.nl/pdf/7.pdf 6. Bevendorff, J., Stein, B., Hagen, M., Potthast, M.: Elastic chatnoir: Search engine for the clueweb and the common crawl. In: Pasi, G., Piwowarski …


Multilingual is not enough: BERT for Finnish

A Virtanen, J Kanerva, R Ilo, J Luoma, J Luotolahti… - arXiv preprint arXiv …, 2019

… Second, we selected texts from the Common Crawl project6 by running aa map-reduce language detection job on the plain text material from Common Crawl. These sources were supplemented with plain text extracted …


Multilingual Sentence-Level Bias Detection in Wikipedia

… Same BOW n-gram size and BOW size and value type as SGD. 5Available for 157 languages, pretrained on Common Crawl and Wikipedia (Grave et al., 2018) https:// fasttext.cc/docs/en/crawl-vectors.html 6Version 0.21.2 of the sklearn toolkit …


Multimodal deep networks for text and image-based document classification

N Audebert, C Herold, K Slimani, C Vidal - APIA

… For both methods, we use the SpaCy small English model [33] to perform the tokenization and punctuation removal. Individual word embeddings are then inferred using FastText [29] pretrained on the Common Crawl dataset …


Multimodal Machine Translation with Embedding Prediction

T Hirasawa, H Yamagishi, Y Matsumura, M Komachi - arXiv preprint arXiv …, 2019

… model. “+ pretrained” models are initialized with pretrained embeddings. 2018). These word embeddings are trained on Wikipedia and Common Crawl using the CBOW algorithm, and the dimension is 300. The embedding …


Multimodal Sentiment Analysis Using Deep Learning

R Sharma, N Le Tan, F Sadat - 2018 17th IEEE International Conference on Machine …, 2018

… For the CNN model we used pre-trained word embeddings (GloVe 840B.300d). This is a 300-dimensional word embedding trained on 840 billion tokens from the common crawl dataset. The maximum sequence length is 200 …


Named entity recognition for Polish

M Marcińczuk, A Wawer - Poznan Studies in Contemporary Linguistics, 2019

AbstractIn this article we discuss the current state-of-the-art for named entity recognition for Polish. We present publicly available resources and open-source tools for named entity recognition. The overview includes various …


Named Entity Recognition for Social Media Text

… We use two different pre-trained word embeddings based on Common Crawl data, which contains 840 billion tokens and 2.2 million vocabulary and Twitter data which contains 2 billion tweets, 27 billion tokens, and 1.2 million vocabulary …


Named Entity Recognition Using Gazetteer of Hierarchical Entities

M Štravs, J Zupančič - … Conference on Industrial, Engineering and Other …, 2019

… To summarize, the proposed entity recognition method was tested using two languages (Slovenian and English), six different distance measures, and two different vector embeddings from Wikipedia (Wiki WV) and Common Crawl (CC WV) …


Named-entity recognition in Czech historical texts: Using a CNN-BiLSTM neural network model

… We also tried to work with published pretrained word embeddings of contemporary Czech words provided by fastText6. These were trained on more than 178 millions of tokens from Wikipedia and 13 billions tokens based on common crawl (Grave et al., 2018) …


Page 1. Natural Language Processing for Book Recommender Systems by Haifa Alharthi Thesis submitted in partial fulfillment of the requirements for the PhD degree in Computer Science School of Electrical Engineering and Computer Science Faculty of Engineering …


Natural language processing using context-specific word vectors

B McCann, C Xiong, R Socher - US Patent App. 15/982,841, 2018

… in the second language. In some examples, training of an MT-LSTM of the encoder 310 uses fixed 300-dimensional word vectors, such as the CommonCrawl-840B GloVe model for English word vectors. These word vectors …


Naver Labs Europe's Systems for the WMT19 Machine Translation Robustness Task

A Bérard, I Calapodescu, C Roux - arXiv preprint arXiv:1907.06488, 2019

… 3.1 Pre-processing CommonCrawl filtering We first spent efforts on filtering and cleaning the WMT data (in particular CommonCrawl) … We filtered CommonCrawl as follows: we trained a baseline FR→EN model on WMT without …


T Trinh, T Quan, T Mai - arXiv preprint arXiv:1905.00195, 2019

Page 1. Noname manuscript No. (will be inserted by the editor) Nested Variational Autoencoder for Topic Modeling on Microtexts with Word Vectors Trung Trinh · Tho Quan · Trung Mai Received: date / Accepted: date Abstract …


NeuMorph: Neural Morphological Tagging for Low-Resource Languages—An Experimental Study for Indic Languages

A Chakrabarty, A Chaturvedi, U Garain - ACM Transactions on Asian and Low …, 2019

Page 1. 16 NeuMorph: Neural Morphological Tagging for Low-Resource Languages— An Experimental Study for Indic Languages ABHISEK CHAKRABARTY, AKSHAY CHATURVEDI, and UTPAL GARAIN, Indian Statistical Institute, India …


Page 1. Neural Conversation Recommendation with Online Interaction Modeling Xingshan Zeng1,2, Jing Li3∗, Lu Wang4, Kam-Fai Wong1,2 1The Chinese University of Hong Kong, Hong Kong, China 2MoE Key Laboratory …


Page 1. Neural Facet Detection on Medical Resources Thomas Steffek April 2, 2019 Page 2. Page 3. Beuth Hochschule für Technik Fachbereich VI - Informatik und Medien Database Systems and Text-based Information Systems (DATEXIS) Bachelor's thesis …


Neural Feature Extraction for Contextual Emotion Detection

… pretrained word embeddings. As the first word embedder, we chose GloVe (Pennington et al., 2014), which is pretrained on 840B tokens of web data from Common Crawl, and provides 300d vectors as word embeddings. As our sec …


Neural Grammatical Error Correction by Simulating the Human Learner and the Human Proofreader

F Gaim, JW Chung, JC Park - 한국정보과학회 학술발표논문집, 2018

… For this and the contrastive learning, we use a large 5-gram language model trained on the Common Crawl data [8]. Training and Decoding: To effectively handle out-of- vocabulary words, we use sub-word level tokenization and …


Neural Machine Translation for English–Kazakh with Morphological Segmentation and Synthetic Data

A Toral, L Edman, G Yeshmagambetova, J Spenader - … 2: Shared Task Papers, Day 1 …, 2019

… 7.5 0.19 0.16 Wikititles 117.0 0.23 0.19 Table 1: Preprocessed EN–KK parallel training data. Words (M) Corpus Sentences (k) EN RU Common crawl 871.8 20.82 19.97 News-comm … Corpus Threshold Pairs left (k) CommonCrawl 0.7323 568.50 News Comm …


Neural network learning engine

CM Ormerod - US Patent App. 16/286,566, 2019

… skill and not to limit the invention to any one embodiment, commercial word embedding tools can include Google News word embedding, which has been trained on an extensive corpus of news items, and/or GloVe word …


Neural network-based approaches for biomedical relation classification: A review

Y Zhang, H Lin, Z Yang, J Wang, Y Sun, B Xu, Z Zhao - Journal of Biomedical …, 2019

… Word2vec, Google news, https://code.google.com/archive/p/word2vec. GloVe, Wikipedia, Gigaword, Common Crawl, Twitter, https://nlp.stanford. edu/projects/glove. fastText, Wikipedia, UMBC corpus, news corpus …


Neural NLP models under low-supervision scenarios

Page 1. Copyright by Ye Zhang 2019 Page 2. The Dissertation Committee for Ye Zhang certifies that this is the approved version of the following dissertation: Neural NLP Models Under Low-supervision Scenarios Committee: Matthew A Lease, Supervisor …


Neural Text Style Transfer via Denoising and Reranking

J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng - … of the Workshop on Methods for …, 2019

… 3. Fluency The post-transfer sentence should remain grammatical and fluent. We use the average log probability of the sentence posttransfer with respect to a language model trained on CommonCrawl as our measure of fluency …


NLNDE: The Neither-Language-Nor-Domain-Experts' Way of Spanish Medical Document De-Identification

… S2 (FLAIR+fastText): In contrast to all other runs, the second run uses only domain-independent embeddings, ie, embeddings that have been trained on standard narrative and news data from Common Crawl and Wikipedia …


NLP@ UIOWA at SemEval-2019 Task 6: Classifying the Crass using Multi-windowed CNNs

J Rusert, P Srinivasan - Proceedings of the 13th International Workshop on …, 2019

… Word embeddings for Non-Out of Vocabulary (OOV) words are obtained from Glove (Pennington et al., 2014) which has been trained on Twitter data3. Experiments were also conducted with Glove common crawl data, but no visible improvement was found …


Noisy Parallel Corpus Filtering through Projected Word Embeddings

M Kurfalı, R Östling - Proceedings of the Fourth Conference on Machine …, 2019

… Larger monolingual corpora based on Wikipedia and common crawl data were also provided.2 To train our model, we use all the parallel data available for the English-Sinhala and EnglishNepali pairs (summarized …


NRC Parallel Corpus Filtering System for WMT 2019

G Bernier-Colborne, C Lo - Proceedings of the Fourth Conference on Machine …, 2019

… embedding models. Common Crawl data was not used to train the bilingual word embeddings. 2.2 … representation layer. We used XLM to train a model using almost all the available data, except for the monolingual English Common Crawl data. This …


J Cao, M Tanana, ZE Imel, E Poitras, DC Atkins…

Page 1. Observing Dialogue in Therapy: Categorizing and Forecasting Behavioral Codes Jie Cao†, Michael Tanana‡, Zac E. Imel‡, Eric Poitras‡, David C. Atkins♦, Vivek Srikumar† †School of Computing, University of Utah …


Observing LOD Using Equivalent Set Graphs: It Is Mostly Flat and Sparsely Linked

L Asprino, W Beek, P Ciancarini, F van Harmelen… - International Semantic Web …, 2019

… The two largest available crawls of LOD available today are WebDataCommons and LOD-a-lot. WebDataCommons 2 [12] consists of \(\sim \)31B triples that have been extracted from the CommonCrawl datasets (November 2018 version) …


Observing the LOD Cloud using Equivalent Set Graphs: the LOD Cloud is mostly flat and sparsely linked

L Asprino, W Beek, P Ciancarini, F van Harmelen…

… The two largest available crawls of LOD available today are WebDataCommons and LOD-a-lot. WebDataCommons5 [12] consists of ∼31B triples that have been extracted from the CommonCrawl datasets (November 2018 version) …


OECD Analytical Database on Individual Multinationals and their Affiliates (ADIMA)

… Secondly, information from MNE webpages is used from an open source 'copy of the internet' generated via web crawling from the Common Crawl 4 . This process develops a graph of the links between companies, from …


Offensive Language and Hate Speech Detection for Danish

GI Sigurbergsson, L Derczynski - arXiv preprint arXiv:1908.04531, 2019

… sample of text. Pre-trained Embeddings. The pre-trained FastText [24] embeddings are trained on data from the Common Crawl project and Wikipedia, in 157 languages (including English and Danish). FastText also provides …


On extracting data from tables that are encoded using HTML

JC Roldán, P Jiménez, R Corchuelo - Knowledge-Based Systems, 2019


On Implementing the Binary Interpolative Coding Algorithm

… Table 4. Decoding time measured in average nanoseconds spent per decoded integer, for the run-aware implementation (ra) and for the not run-aware implementation. • CCNews is an English subset of the freely available news from CommonCrawl 3, consisting of …


S Dev, T Li, J Phillips, V Srikumar - arXiv preprint arXiv:1908.09369, 2019

Page 1. arXiv:1908.09369v1 [cs.CL] 25 Aug 2019 On Measuring and Mitigating Biased Inferences of Word Embeddings Sunipa Dev, Tao Li, Jeff Phillips, Vivek Srikumar School of Computing University of Utah Abstract Word …


C May, A Wang, S Bordia, SR Bowman, R Rudinger - arXiv preprint arXiv:1903.10561, 2019

Page 1. On Measuring Social Biases in Sentence Encoders Chandler May1 Alex Wang2 Shikha Bordia2 Samuel R. Bowman2 Rachel Rudinger1 1Johns Hopkins University 2New York University {cjmay,rudinger}@jhu.edu {alexwang,sb6416,bowman}@nyu.edu Abstract …


On Optimally Partitioning Variable-Byte Codes

GE Pibiri, R Venturini - IEEE Transactions on Knowledge and Data …, 2019

Page 1. 1041-4347 (c) 2018 IEEE. Personal use is permitted, but republication/ redistribution requires IEEE permission. See http://www.ieee.org/ publications_standards/publications/rights/index.html for more information. This …


Page 1. Jagiellonian University Faculty of Mathematics and Computer Science Theoretical Computer Science Stationary Studies Index number: 1079621 Tomasz Wesołowski On relevance of enriching word embeddings in solving Natural Language Inference problem …


On Slicing Sorted Integer Sequences

GE Pibiri - arXiv preprint arXiv:1907.01032, 2019

… 2009. • CCNews is a dataset of news freely available from CommonCrawl: http://commoncrawl.org/ 2016/10/news-dataset-available. Precisely, the datasets consists of the news appeared from 09/01/16 to 30/03/18. Identifiers …


On the Effect of Low-Frequency Terms on Neural-IR Models

S Hofstätter, N Rekabsaz, C Eickhoff, A Hanbury - arXiv preprint arXiv:1904.12683, 2019

… collection. The details of the resulting 1Provided in the form of evaluation tuples: top1000.dev.tsv 242B lower-cased (CommonCrawl) from: https://nlp.stanford.edu/ projects/glove/ Table 1: Left: Details of the vocabularies. Right …


On the Robustness of Unsupervised and Semi-supervised Cross-lingual Word Embedding Learning

Y Doval, J Camacho-Collados, L Espinosa-Anke… - arXiv preprint arXiv …, 2019

… google.com/site/rmyeid/projects/polyglot 2The sources of the web-corpora are: UMBC (Han et al., 2013), 1-billion (Cardellino, 2016), itWaC and sdeWaC (Ba- roni et al., 2009), Hamshahri (AleAhmad et al., 2009), and Common Crawl downloaded from http://www …


On Using Machine Learning to Identify Knowledge in API Reference Documentation

D Fucci, A Mollaalizadehbahnemiri, W Maalej - arXiv preprint arXiv:1907.09807, 2019

… For the deep learning classifiers in our benchmark, we train GloVe [19] embeddings based on four large corpora, summarized in Table 3. The Common Crawl (CC) is a pre-trained embedding downloaded in …


On Using SpecAugment for End-to-End Speech Translation

… For MT training, we use the TED, and the OpenSubtitles2018 corpora, as well as the data provided by the WMT 2018 evaluation (Europarl, ParaCrawl, CommonCrawl, News Commentary, and Rapid), a total of 65M lines of parallel sentences …


One Epoch Is All You Need

A Komatsuzaki - arXiv preprint arXiv:1906.06669, 2019

… Trinh & Le (2018) pointed out that CommonCrawl contains a large portion of corrupt samples, which makes it unsuitable for the training. The proportion of the corrupt samples in CommonCrawl is substantially higher than 50 …


Page 1. Universität des Saarlandes Master's Thesis Online Parallel Data Extraction with Neural Machine Translation submitted in fulfillment of the degree requirements of the MSc in Language Science and Technology at Saarland University …


Page 1. Ontological Traceability using Natural Language Processing A master thesis presented by Edder de la Rosa Benitez Submitted to the Department of Organization and Information in partial fulfillment of the …


OpenCeres: When Open Information Extraction Meets the Semi-Structured Web

… 5.1 Experimental Setup Datasets: Our primary dataset is the augmented SWDE corpus described in Section 4. In addition, we used the set of 315 movie websites (comprising 433,000 webpages) found in Common …


OPTIMIZE THE LEARNING RATE OF NEURAL ARCHITECTURE IN MYANMAR STEMMER

… Word vector pre-trained on large text corpora have been released on [10] "Learning Word Vectors for 157 Languages " that trained on 3 billion words from Wikipedia and Common Crawl using Continuous bag-of-words (CBOW) 300-dimension …


Optimizer Comparison with Dropout for Neural Sequence Labeling in Myanmar Stemmer

O Yadanar, KM Soe - 2019 IEEE International Conference on Industry 4.0 …, 2019

… Parameter initialization: It has used Learning Word Vectors for 157 Languages that trained on 3 billion words from Wikipedia and Common Crawl using CBOW 300-dimension (E. Grave, P. Bojanowski, P. Gupta, A. Joulin, T. Mikolov,2018) for both word and character …


Optimizing Social Media Data Using Genetic Algorithm

S Das, AK Kolya, D Das - Metaheuristic Approaches to Portfolio Optimization, 2019

Page 1. 126 Copyright © 2019, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. Chapter 6 DOI: 10.4018/978-1-5225-8103-1.ch006 ABSTRACT Twitter-based …


Overview of the CLEF eHealth Evaluation Lab 2019

E Kanoulas, D Li, L Azzopardi, R Spijker, G Zuccon… - Experimental IR Meets …

… More specifically, for the Abstract and Title Screening subtask the PubMed Document Identifiers (PMIDs) of potentially relevant 4http://commoncrawl. org/(last accessed on 28 May 2019) … It consists of web pages acquired from the CommonCrawl …


P-SIF: Document Embeddings Using Partition Averaging

V Gupta, A Saw, P Nokhiz, P Netrapalli, P Rai…

… Page 5. evaluation. We use the PARAGRAM-SL999 (PSL) as word embeddings, obtained by training on the PPDB dataset. 7 We use the fixed weighting parameter a value of 10−3, and the word frequencies p(w) are estimated from the commoncrawl dataset …


P2L: Predicting Transfer Learning for Images and Semantic Relations

B Bhattacharjee, N Codella, JR Kender, S Huo… - arXiv preprint arXiv …, 2019

… We use the CC-DBP [12] dataset: the text of Common Crawl1 and the semantic relations schema and training data from DBpedia [1]. DBpedia is a knowledge graph extracted from the infoboxes from Wikipedia … 4.3.2 Validation on Common Crawl - DBpedia …


PaDAWaNS

Page 1. PaDAWaNS Proactive Domain Abuse Warning and Notification System by TLM Brands to obtain the degree of Master of Science at the Delft University of Technology, to be defended publicly on Tuesday January 15, 2019 at 11:00 AM …


Parallel External Memory Wavelet Tree and Wavelet Matrix Construction

J Ellert, F Kurpicz - International Symposium on String Processing and …, 2019

… CC \((\sigma =242)\) contains websites (without HTML tags) that have been crawled by the Common Crawl corpus (http://commoncrawl.org), and. Wiki \((\sigma =213)\) are recent Wikipedia dumps containing XML files that …


Paraphrase-Sense-Tagged Sentences

A Cocos, C Callison-Burch, S Chen, D Khashabi… - Transactions, 2019


PDRCNN: Precise Phishing Detection with Recurrent Convolutional Neural Networks

W Wang, F Zhang, X Luo, S Zhang - Security and Communication Networks, 2019

… This method first encodes the URL string using the one-hot encoding method, and then inputs each encoded character vector into the LSTM neurons for training and testing. The method achieved an accuracy of 0.935 on the …


Peer Review and the Production of Scholarly Knowledge: Automated Textual Analysis of Manuscripts Revised for Publication in Administrative Science Quarterly

D Strang, F Dokshin - The Production of Managerial Knowledge and …, 2019

… numbers, and filter out “stop words.” Stop words are the most common words in the English language (eg, “the,” “not,” “a”). 2 Next, for each word in the pre-processed sentences, we generate word vectors from a GloVe model …


PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization

J Zhang, Y Zhao, M Saleh, PJ Liu - arXiv preprint arXiv:1912.08777, 2019

… T5 (Raffel et al., 2019) generalized the text-to- text framework to a variety of NLP tasks and showed the advantage of scaling up model size (to 11 billion parameters) and pre-training corpus, introducing C4, a massive text corpus …


Page 1. Running head: MENTAL STATE DIMENSIONS 1 People represent mental states in terms of rationality, social impact, and valence: Validating the 3d Mind Model Mark A. Thornton* and Diana I. Tamir Department of …


PhishFry–A Proactive Approach to Classify Phishing Sites using SCIKIT Learn

… [Online]. Available: http://5000best.com/websites/. [Accessed 2019]. [26] OpenPhish, "OpenPhish," 2019. [Online]. Available: https://openphish.com/. [27] Amazon Web Services, "Common Crawl," Amazon, 2019. [Online] …


Phishing Detection Based on Machine Learning and Feature Selection Methods

M Almseidin, AMA Zuraiq, M Al-kasassbeh, N Alnidami - International Journal of …, 2019

… Phishing webpages are collected from Phish-Tank and Open-Phish, while legitimate web-pages are collected from Alexa and Common Crawl. These web-pages are downloaded on two distinct sessions, from January to May 2015 and through May to June 2017 …


Phishing URL Detection Via Capsule-Based Neural Network

Y Huang, J Qin, W Wen - 2019 IEEE 13th International Conference on Anti …, 2019

… [27] VirusTotal, https://www.virustotal.com/ [28] Common Crawl, https://commoncrawl.org/ [29] J. Ma, LK Saul, S. Savage, and GM VoelNer, “Beyond blacNlists: learning to detect malicious web sites from suspicious …


Y Kim, P Petrov, P Petrushkov, S Khadivi, H Ney - arXiv preprint arXiv:1909.09524, 2019

Page 1. Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages Yunsu Kim1∗ Petre Petrov1,2∗ Pavel Petrushkov2 Shahram Khadivi2 Hermann Ney1 1RWTH Aachen University, Aachen …


PKUSE at SemEval-2019 Task 3: Emotion Detection with Emotion-Oriented Neural Attention Network

L Ma, L Zhang, W Ye, W Hu - Proceedings of the 13th International Workshop on …, 2019

… Table 1: Datasets for Semeval-2019 Task 3. 4.2 Experiments The model is implemented using Keras 2.0 (Chollet et al., 2017). We experiment with Stanford's GloVe 300 dimensional word embeddings trained on 840 billion words from Common Crawl …


PLAGO: A SYSTEM FOR PLAGIARISM DETECTION AND INTERVENTION IN MASSIVE COURSES

… Web Crawl: Used for queuing and monitoring of importing web pages from the CommonCrawl.org public dataset (described in 3.5.2). • Admin Options … pages. Common Crawl is a non-profit organization which offers a public …


Page 1. University of Massachusetts Amherst ScholarWorks@UMass Amherst Doctoral Dissertations Dissertations and Theses 2019 Poetry: Identification, Entity Recognition, and Retrieval John J. Foley IV Follow this and additional …


Populating Web Scale Knowledge Graphs using Distantly Supervised Relation Extraction and Validation

A Gliozzo, MR Glass, S Dash, M Canim - arXiv preprint arXiv:1908.08104, 2019

… Also, a web-scale experiment conducted to extend DBPedia with knowledge from Common Crawl shows that our system is not only scalable but also does not require any adaptation cost, while yielding substantial accuracy gain. 1 Introduction …


Precise Detection of Content Reuse in the Web

C Ardi, J Heidemann - ACM SIGCOMM Computer Communication Review, 2019

… We verify our algorithm and its choices with controlled experiments over three web datasets: Common Crawl (2009/10), GeoCities (1990s–2000s), and a phishing corpus (2014) … In the Common Crawl dataset of 40.5×109 chunks, we set the threshold to 105 …


Predicting ConceptNet Path Quality Using Crowdsourced Assessments of Naturalness

Y Zhou, S Schockaert, JA Shah - arXiv preprint arXiv:1902.07831, 2019

… The number in parenthesis after each feature name indicates the dimension of that feature. Vertex embedding (300) This feature is taken directly from the 300dimensional GloVe (25) embedding, pre-trained on the Common Crawl2 dataset with 840 billion tokens …


Predicting Word Concreteness and Imagery

J Charbonnier, C Wartena - Proceedings of the 13th International Conference on …, 2019

… The other two version (also available with and without subword information) with 2 million word vectors trained on the Common Crawl with 600B tokens. In our experiments we used the version trained on Common Crawl without …


Probing Contextualized Sentence Representations with Visual Awareness

Z Zhang, R Wang, K Chen, M Utiyama, E Sumita… - arXiv preprint arXiv …, 2019

… We used newsdev2016 as the dev set and newstest2016 as the test set. 2) For the EN-DE translation task, 4.43M bilingual sentence pairs of the WMT14 dataset were used as training data, including Common Crawl, News Commentary, and Europarl v7 …


Product Classification Using Microdata Annotations

Z Zhang, M Paramita - International Semantic Web Conference, 2019

… dimension of the continuous vector representation of each word. In this work, we use the GloVe word embedding vectors pre-trained on the Common Crawl corpus 3 with 300 dimensions. Since we are dealing with content from e …


Provision and Usage of Provenance Data in the WebIsALOD Knowledge Graph

S Hertling, H Paulheim - CEUR Workshop Proceedings, 2018

… As described in [6], the Copyright c 2018 for this paper by its authors. Copying permitted for private and academic purposes. 1 https://commoncrawl.org 2 NP stands for noun phrase. 3 https://www.w3.org/TR/skos-reference/ Page 2. isa:concept/_Gmail …


PT-CoDE: Pre-trained Context-Dependent Encoder for Utterance-level Emotion Recognition

W Jiao, MR Lyu, I King - arXiv preprint arXiv:1910.08916, 2019

… Here, we utilize the 300-dimensional pre-trained GloVe word vectors1 (Pennington et al., 2014) trained over 840B Common Crawl to initialize the word embedding layer. Those words that cannot be found in the GloVe …


QE BERT: Bilingual BERT using Multi-task Learning for Neural Quality Estimation

H Kim, JH Lim, HK Kim, SH Na - Proceedings of the Fourth Conference on Machine …, 2019

… We used parallel data provided for the WMT19 news machine translation task6 to pre-train QE BERT. The English-Russian parallel data set consisted of the ParaCrawl corpus, Common Crawl corpus, News Commentary corpus, and Yandex …


QED: A Fact Verification and Evidence Support System

… embedding layers, as described below. 4.3.2 Embedding We use GloVe word embeddings (Pennington et al., 2014) with 300 dimensions pretrained using CommonCrawl to get a vector representation of the evidence sentence. We


Quantifying the Semantic Core of Gender Systems

… 4The FASTTEXT word embeddings were trained using Common Crawl and Wikipedia data, using CBOW with po- sition weights, with character n-grams of length 5. For more information, see http://fasttext.cc/docs/en …


QuAVONet: Answering Questions on the SQuAD Dataset with QANet and Answer Verifier

… 5.2 Implementation Details For the word embeddings, I used the starter code's 300-dimensional GloVE vectors trained on the CommonCrawl dataset [6]. These embeddings remained unchanged and were not trained for any of my models …


Quick and (maybe not so) Easy Detection of Anorexia in Social Media Posts

… As shown in Figure 1, these token vectors are then fed to the hidden layer. Two different pretrained word embeddings were experimented with. The first word embedder was the 300d version of GloVe [26] that was pretrained …


Quotient Hash Tables-Efficiently Detecting Duplicates in Streaming Data

R Géraud, M Lombard-Platet, D Naccache - arXiv preprint arXiv:1901.04358, 2019

Page 1. arXiv:1901.04358v1 [cs.DS] 14 Jan 2019 Quotient Hash Tables - Efficiently Detecting Duplicates in Streaming Data Rémi Gérauda,c, Marius Lombard-Platet∗ a,b, and David Naccachea,c aDépartement d'informatique …


Racial bias in legal language

D Rice, JH Rhodes, T Nteta - Research & Politics, 2019

Although racial bias in the law is widely recognized, it remains unclear how these biases are in entrenched in the language of the law, judicial opinions. In th...


Random Projection in Deep Neural Networks

PI Wójcik - arXiv preprint arXiv:1812.09489, 2018

Page 1. Akademia Górniczo-Hutnicza im. Stanisława Staszica w Krakowie Wydział Informatyki, Elektroniki i Telekomunikacji Katedra Informatyki Rozprawa doktorska Zastosowania metody rzutu przypadkowego w głębokich …


Real or Fake? Learning to Discriminate Machine from Human Generated Text

A Bakhtin, S Gross, M Ott, Y Deng, MA Ranzato… - arXiv preprint arXiv …, 2019

… CCNews: We collect a de-duplicated subset of the English portion of the CommonCrawl news dataset [Nagel, 2016], which totals around 16 Billion words … Sebastian Nagel. Cc-news. http://web.archive.org/save/http …


Real-time Claim Detection from News Articles and Retrieval of Semantically-Similar Factchecks

B Adler, G Boscaini-Gilroy - arXiv preprint arXiv:1907.02030, 2019

… new problem. Many unsupervised text embeddings are trained on the CommonCrawl 1 dataset of approx. 840 billion tokens. This … dataset. Supervised datasets are 1CommonCrawl found at http://commoncrawl.org/ unlikely ever …


Real-time event detection using recurrent neural network in social sensors

VQ Nguyen, TN Anh, HJ Yang - International Journal of Distributed Sensor Networks, 2019

We proposed an approach for temporal event detection using deep learning and multi-embedding on a set of text data from social media. First, a convolutional neural network augmented with multiple w...


Real-world Conversational AI for Hotel Bookings

B Li, N Jiang, J Sham, H Shi, H Fazal - arXiv preprint arXiv:1908.10001, 2019

… We compare the following models: 1) Averaged GloVe + feedforward: We use 100dimensional, trainable GloVe embeddings [17] trained on Common Crawl, and produce sentence embeddings for each of the two inputs by averaging across all tokens …


Recommendation System with Aspect-Based Sentiment Analysis

… The word vectors model we use is the "en_core_web_lg" model in spaCy. The model contains English multi-task CNN trained on OntoNotes 5[3], with GloVe[8] vectors trained on Common Crawl. It provides 300dimensional …


Refining Word Reesprentations by Manifold Learning

C Yonghe, H Lin, L Yang, Y Diao, S Zhang, F Xiaochao

… judgment. This is exemplified by the WS353[Finkelstein et al., 2001]word similarity ground truth in Figure 1. Based on the Common Crawl corpus (42B), the Glove model is used to train 300-dimensional word vectors. The similarity …


Regressing Word and Sentence Embeddings for Regularization of Neural Machine Translation

IJ Unanue, EZ Borzeshi, M Piccardi - arXiv preprint arXiv:1909.13466, 2019

… De-En: The German-English dataset (de-en) has been taken from the WMT18 news translation shared task1. The training set contains over 5M sentence pairs collected from the Europarl, CommonCrawl and Newscommentary parallel corpora …


Rel4KC: A Reinforcement Learning Agent for Knowledge Graph Completion and Validation

… The fact triples extracted from free text (Common Crawl) are then fed to the trained RL agent to determine their trustworthiness. If passing the validation, a triple is entered into target KG … The free text used in this study is Common Crawl corpus …


Repositioning privacy concerns: Web servers controlling URL metadata

R Ferreira, RL Aguiar - Journal of Information Security and Applications, 2019

… on empirical observation of web browsers and HTTP server implementations, and while some implementations allow longer URLs (eg, 100.000 octets) this value remains a reasonable assumption for practical purposes 1 . Our …


Representation Learning for Question Classification via Topic Sparse Autoencoder and Entity Embedding

D Li, J Zhang, P Li - IEEE Big Data, 2018

… WordNet 2. The embeddings of entity-related information are also trained with skip-gram. The word embeddings are initialized with the 300 dimensional pretrained vectors 3 from the Common Crawl of 840 billion tokens and 2.2 …


Representing Overlaps in Sequence Labeling Tasks with a Novel Tagging Scheme: bigappy-unicrossy

… a language-independent system based on the bidirectional LSTM-CRF model provided by [7]. Similar to Deep-BGT system [2], we make use of the pretrained word embeddings provided by fastText [6]. The word embeddings …


Review and Visualization of Facebook's FastText Pretrained Word Vector Model

JC Young, A Rusli - … International Conference on Engineering, Science, and …, 2019

… Machine Learning (ML). Currently, FastText provides pretrained Word2Vec model for 157 language that trained on Common Crawl and Wikipedia (Bahasa Indonesia is one from the provided model) [15]. In its Word2Vec model …


RIPPED: Recursive Intent Propagation using Pretrained Embedding Distances

… GloVe (Pennington et al., 2014) is a word embedding model trained on data from the Common Crawl corpus6. GloVe is a log-bilinear regression model that incorporates both local context windows and global matrix …


RNN Embeddings for Identifying Difficult to Understand Medical Words

H Pylieva, A Chernodub, N Grabar, T Hamon - … of the 18th BioNLP Workshop and …, 2019

… improve classification accuracy for our specific problem. We note that FastText word embeddings trained on Wikipedia and Common Crawl5 texts have an important part of words from our dataset. According to our analysis, the …


RoBERTa: A Robustly Optimized BERT Pretraining Approach

Y Liu, M Ott, N Goyal, J Du, M Joshi, D Chen, O Levy… - arXiv preprint arXiv …, 2019

… (16GB). • CC-NEWS, which we collected from the En- glish portion of the CommonCrawl News dataset (Nagel, 2016) … STORIES, a dataset introduced in Trinh and Le (2018) containing a subset of CommonCrawl data filtered …


Robust Argument Unit Recognition and Classification

D Trautmann, J Daxenberger, C Stab, H Schütze… - arXiv preprint arXiv …, 2019

… 2http://commoncrawl.org/2016/02/ february-2016-crawl-archive-now-available/ 3https://www.elastic.co/products/ elasticsearch the topic. Each document was checked for its corresponding WARC file at the Common Crawl In …


Robust Named Entity Recognition with Truecasing Pretraining

S Mayhew, N Gupta, D Roth - arXiv preprint arXiv:1912.07095, 2019

… and Kauchak (2011) and used in Susanto, Chieu, and Lu (2016), and a specially preprocessed large dataset from English Common Crawl (CC).1 … 1commoncrawl.org 2In a naming clash, the moses script is called …


SACABench: Benchmarking Suffix Array Construction

J Bahne, N Bertram, M Böcker, J Bode, J Fischer… - International Symposium on …, 2019

… We removed every character but A, C, G, and T. CommonCrawl (\(\sigma =242,\mathrm {avg\_lcp}=3,995, \mathrm {max\_lcp}=605,632\)), which is a crawl of the web done by the CommonCrawl Corpus (http://commoncrawl.org) without any HTML tags …


Samsung and University of Edinburgh's System for the IWSLT 2019

J Wetesko, M Chochowski, P Przybysz, P Williams… - 2019

… CommonCrawl and NewsCrawl corpora we used the approach de- scribed in [5]. Two RNN language models were constructed using Marian toolkit: in-domain trained with MUST-C corpus and out-of-domain created using …


C Fu, C Wang, D Cai - arXiv preprint arXiv:1907.06146, 2019

Page 1. Satellite System Graph: Towards the Efficiency Up-Boundary of Graph-Based Approximate Nearest Neighbor Search Cong Fu, Changxu Wang, Deng Cai ∗ The State Key Lab of CAD&CG, College of Computer Science …


SberQuAD--Russian Reading Comprehension Dataset: Description and Analysis

P Efimov, L Boytsov, P Braslavski - arXiv preprint arXiv:1912.09723, 2019

… We tokenized text using spaCy16. To initialize the embedding layer for BiDAF, DocQA, DrQA, and R-Net we use Russian case-sensitive fastText embeddings trained on Common Crawl and Wikipedia17. This initialization is used for both questions and paragraphs …


SC-UPB at the VarDial 2019 Evaluation Campaign: Moldavian vs. Romanian Cross-Dialect Topic Identification

C Onose, DC Cercel, S Trausan-Matu - Proceedings of the Sixth Workshop on NLP …, 2019

… (2018), Nordic Language Processing Laboratory (NLPL) word embedding repository (Kutuzov et al., 2017) and Common Crawl (CC) word vectors (Grave et al., 2018). The relevant details for each word vector representation model can be viewed in Table 2 …


Scalable Cross-Lingual Transfer of Neural Sentence Embeddings

H Aldarmaki, M Diab - arXiv preprint arXiv:1904.05542, 2019

… We used WMT'12 Common Crawl data for crosslingual alignment, and WMT'12 test sets for evaluations. We used the augmented SNLI data de- scribed in (Dasgupta et al., 2018) and their translations for training the mono-lingual and joint InferSent models …


K KS, S Sangeetha - arXiv preprint arXiv:1903.01039, 2019

KK Subramanyam, S Sivanesan - Journal of Biomedical Informatics, 2019

Page 1. 1 SECNLP: A Survey of Embeddings in Clinical Natural Language Processing Kalyan KS, S. Sangeetha Text Analytics and Natural Language Processing Lab Department of Computer Applications National …


Security In Plain TXT

… These seed domains are compiled from a combination of sources, including the Alexa top 1 million, the TLD zone files for COM, NAME, NET, ORG, and BIZ, sites captured by the Common Crawl project, multiple public domain …


Page 1. Page 2. Page 3. Security Posture Based Incident Forecasting A Thesis Submitted to the Faculty of Drexel University by Dagmawi Mulugeta in partial fulfillment of the requirements for the degree of Master of Science June 2019 Page 4 …


See-Through-Text Grouping for Referring Image Segmentation

DJ Chen, S Jia, YC Lo, HT Chen, TL Liu - … of the IEEE International Conference on …, 2019

… The representation st is visual-attended and its goodness is linked to the predicted segmentation map Pt−1. The GloVe model in our implementation is pre-trained on Common Crawl in 840B tokens. Following …


Semantic Characteristics of Schizophrenic Speech

K Bar, V Zilberstein, I Ziv, H Baram, N Dershowitz… - arXiv preprint arXiv …, 2019

… Specifically, we used Hebrew pretrained vectors provided by fastText (Grave et al., 2018), which were created from Wikipedia,3 as well as from other content extracted from the web with Common Crawl.4 Overall, 97% of the words in our corpus exist in fastText …


Semantic similarity measure for Thai language

… In this paper, pre-trained word vectors from fastText [10] and Thai2vec [1] corpus are used to compute the similarity between given words. The facebook research distributed the word vector trained on a common crawl and Wikipedia using the fastText model …


Semantic Textual Similarity Measures for Case-Based Retrieval of Argument Graphs

M Lenz, S Ollinger, P Sahitaj, R Bergmann - International Conference on Case-Based …, 2019

… Word2vec GoogleNews 3 vectors are trained on the Google News dataset on about 100B tokens. GloVe 4 is trained on the Common Crawl dataset on 840B tokens. fastText 5 vectors are trained on Wikipedia and Common Crawl …


H Martindale, E Rowland, T Flower - 16th Meeting of the Ottawa Group on Price …, 2019

Page 1. Office for National Statistics 1 Semi-supervised machine learning with word embedding for classification: April 2019 26/04/2019 Semi-supervised machine learning with word embedding for classification in price statistics …


Semi-supervised Neural Machine Translation via Marginal Distribution Estimation

Y Wang, Y Xia, L Zhao, J Bian, T Qin, E Chen, TY Liu - IEEE/ACM Transactions on …, 2019

Page 1. 2329-9290 (c) 2019 IEEE. Personal use is permitted, but republication/ redistribution requires IEEE permission. See http://www.ieee.org/ publications_standards/publications/rights/index.html for more information. This …


SENPAI: Supporting Exploratory Text Analysis through Semantic & Syntactic Pattern Inspection

… lemmatization, so as to remove surface form variations which do not alter the meaning of a word, eg the lemma for both “moved” and “moves” is “move.” Then, we encode lemmas with the corresponding 300-dimensional word …


Sense disambiguation for Punjabi language using supervised machine learning techniques

… The character n-grams of length 5 have been applied to words in window of size 5 with 10 negative samples [10]. It has been trained on the Punjabi Wikipedia and the raw web data fetched by common crawl method. 6 Working of WSD System for Punjabi language …


Page 1. Sentence and Word Weighting for Neural Machine Translation Domain Adaptation Pinzhen (Patrick) Chen Undergraduate Dissertation Artificial Intelligence and Software Engineering School of Informatics The …


Sentence Classification and Information Retrieval for Petroleum Engineering

TF Ferraz, GABA Ferreira, FG Cozman, I Santos

… Accordingly, we used a word embedding representation in order to represent the words as vectors and then be able to define and compute distances between terms. We used a pre-trained embedding model called ”Common Crawl” [Pennington et al. 2014] …


Sentence Mover's Similarity: Automatic Evaluation for Multi-Sentence Texts

… We obtain GloVe embeddings, which are type-based, 300-dimensional embeddings trained on Common Crawl,9 using spaCy,10 while the ELMo em- beddings are character-based, 1,024-dimensional, contextual …


Sentence-Level Content Planning and Style Specification for Neural Text Generation

X Hua, L Wang - arXiv preprint arXiv:1909.00734, 2019

… Statistics are shown in Table 1. Input Keyphrases and Label Construction. To obtain the input keyphrase candidates and their sentence-level selection labels, we first construct queries to retrieve passages from Wikipedia and news articles collected from commoncrawl …


Sentiment Analysis

D Sarkar - Text Analytics with Python, 2019

In this chapter, we cover one of the most interesting and widely used aspects pertaining to natural language processing (NLP), text analytics, and machine learning. The problem at hand is sentiment...


Separate Chaining Meets Compact Hashing

D Köppl - arXiv preprint arXiv:1905.00163, 2019

Page 1. Separate Chaining Meets Compact Hashing Dominik Köppl Department of Informatics, Kyushu University, Japan Society for Promotion of Science Abstract While separate chaining is a common strategy for resolving …


Sequence Labeling to Detect Stuttering Events in Read Speech

S Alharbi, M Hasan, AJH Simons, S Brumfitt, P Green - Computer Speech & …, 2019

… In the present study, we used a pre-trained GloVe model to generate word embeddings for each utterance. This model was trained on the Common Crawl (CC) corpus (1.9 M vocab) Pennington et al. (2014). 6. Automatic Speech Recognition System …


Sequence Time Expression Recognition in the Spanish Clinical Narrative

A Ruiz-de-la-Cuadra, JL López-Cuadrado… - 2019 IEEE 32nd …, 2019

… embedding (Table 1). Name Training Words Size Resource Glo200Ve Non-zero entries [37] 840 B 300 Common Crawl Spanish Billion Word [38] Word2Vec [39] 1.5 B 300 Sensem, Ancora Corpus, OPUS Project, etc. EVEX Word2Vec …


Sequence-to-sequence Pre-training with Data Augmentation for Sentence Rewriting

Y Zhang, T Ge, F Wei, M Zhou, X Sun - arXiv preprint arXiv:1909.06002, 2019

… Specifically, for a correct sentence, a back translation model trained with the public GEC data first generates 10 best outputs; then a 5-gram language model (JunczysDowmunt and Grundkiewicz, 2016) trained on Common …


Sequential Attention-based Network for Noetic End-to-End Response Selection

Q Chen, W Wang - arXiv preprint arXiv:1901.02609, 2019

… Embedding Training corpus #Words glove.6B.300d Wikipedia + Gigaword 0.4M glove.840B.300d Common Crawl 2.2M glove.twitter.27B.200d Twitter 1.2M … 1.0M crawl-300d-2M.vec Common Crawl 2.0M word2vec.300d Linux manual pages 0.3M …


Sequential Matching Model for End-to-end Multi-turn Response Selection

Q Chen, W Wang - ICASSP 2019-2019 IEEE International Conference on …, 2019

… Re- sults on the Ubuntu development set are shown in Table 3. We can see that word2vec embedding trained on the training dataset achieves better results than Fasttext [23] embedding trained on the unlabeled corpus …


Sequential transfer learning in NLP for text summarization

… With W and ˜W, the model generates two sets of word vectors which are supposed to perform equally if X is symmetric [64]. The GloVe model has been trained on varying sized datasets from one up to 42 billion (Common Crawl) tokens of data …


Should John Be More Likely A Physician Than Lisa: Bias-Performance Trade-Off for Gendered Pronoun Resolution

… the female gendered words. For our case, we are using the pre-trained Glove6 (these cotain 840B tokens and are trained on the Common Crawl corpus) embeddings to get the hard-debiased em- beddings. To obtain these …


Similarity Driven Approximation for Text Analytics

G Hu, Y Zhang, S Rigo, TD Nguyen - arXiv preprint arXiv:1910.07144, 2019

… For example, the Google Books Ngram data set contains 2.2 TB of data [1], and the Common Crawl corpus contains petabytes of data [2]. Processing such large text data sets can be computationally expensive, especially if it involves sophisticated algorithms …


Situating Sentence Embedders with Nearest Neighbor Overlap

LH Lin, NA Smith - arXiv preprint arXiv:1909.10724, 2019

… GloVe average 100 Wikipedia 2014 + Gigaword 5 (6B tokens, uncased) 300 Wikipedia 2014 + Gigaword 5 (6B tokens, uncased) 300 Common Crawl (840B tokens, cased) FastText average 300 Wikipedia + UMBC + statmt.org …


Six dimensions describe action understanding: the ACT-FASTaxonomy

MA Thornton, D Tamir, PS Hall - PsyArXiv. June, 2019

… different algorithm. For the present purposes, we used a pre-trained version of GloVe based on the Common Crawl: a set of 840 billion tokens generated by scraping the entire web. For model comparison, we derived an …


SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

H Jiang, P He, W Chen, X Liu, J Gao, T Zhao - arXiv preprint arXiv:1911.03437, 2019

… For example, the well-known “Common Crawl project” is producing text data extracted from web pages at a rate of about 20TB per month. The resulting extremely large text corpus allows us to train extremely large neural network-based general language models …


Social Relation Extraction from Chatbot Conversations: A Shortest Dependency Path Approach

M Glas - SKILL 2019-Studierendenkonferenz Informatik, 2019

… The dictionary used here, 5 https://github.com/zalandoresearch/flair 6 https://spacy.io/ 7 http://commoncrawl.org/ 8 https://catalog.ldc.upenn.edu/LDC2013T19 Page 8. 8 Markus Glas Fig. 3: Example of a dependency path within a sentence containing two entities …


Social Sensing for Improving the User Experience in Orienteering

F Persia, S Helmer, S Pugacs, G Pilato - 2019 IEEE 13th International Conference on …, 2019

… ing. In particular, we have used the spaCy “en core web md” language model, which is an “English multi-task Convolutional Neural Network trained on OntoNotes [34], with GloVe [35] vectors trained on Common Crawl [36]” …


A Das, S Baki, AE Aassal, R Verma, A Dunbar - arXiv preprint arXiv:1911.00953, 2019

Page 1. REEXAMINING PHISHING RESEARCH 1 SOK: A Comprehensive Reexamination of Phishing Research from the Security Perspective Avisha Das, Shahryar Baki, Ayman El Aassal, Rakesh Verma, and Arthur Dunbar …


Sparse Victory–A Large Scale Systematic Comparison of Count-Based and Prediction-Based Vectorizers for Text Classification

… Corpus (100 billion words). For greater ease of comparison both the GloVe and fastText models have a dimension of 300 and have been trained on the Common Crawl Corpus (640 billion words). The ELMo embedding has …


ST-Sem: A Multimodal Method for Points-of-Interest Classification Using Street-Level Imagery

SS Noorian, A Psyllidis, A Bozzon - International Conference on Web Engineering, 2019

… representing each word as a bag of character n-grams. We use pre-trained word vectors for 2 languages (English and German), trained on Common Crawl and Wikipedia 6 . According to the detected language l, the corresponding pre …


STAR-GCN: Stacked and Reconstructed Graph Convolutional Networks for Recommender Systems

J Zhang, X Shi, S Zhao, I King - arXiv preprint arXiv:1905.13129, 2019

… For movie features, we concatenate the title name, release year, and one-hot encoded genres. We process title names by averaging the off-the-shelf 300-dimensional GloVe CommonCrawl word vector [Pennington et al., 2014] of each word …


STD: An Automatic Evaluation Metric for Machine Translation Based on Word Embeddings

P Li, C Chen, W Zheng, Y Deng, F Ye, Z Zheng - IEEE/ACM Transactions on Audio …, 2019

… H and M are their means respectively. The word embedding used in our STD implementation is the freely-available fastText word embedding1 [11], which has 2 million word vectors trained on Common Crawl (600B tokens) …


Streaming Infrastructure and Natural Language Modeling with Application to Streaming Big Data

… In our research, we try to find an alternative resource to study such data. Common Crawl is a massive multi-petabyte dataset hosted by Amazon. It contains archived HTML web page data from 2008 to date. Common …


Structured Two-Stream Attention Network for Video Question Answering

L Gao, P Zeng, J Song, YF Li, W Liu, T Mei, HT Shen - Proceedings of the AAAI …, 2019

… consisting of M words, is first converted into a sequence Q = {qm}M m=1, where qm is a one-hot vector representing the word at position m. Next, we employ the word embedding GloVe (Pennington, Socher, and Manning …


Study of Tibetan Text Classification based on fastText

W Ma, H Yu, J Ma - 3rd International Conference on Computer Engineering …

… Every single text in all data is a line, and the "__label__ + tag" is added at the beginning of each line. Pre-training data set: fastText publishes word vectors in 157 languages [13], which are trained on Common Crawl and Wikipedia using fastText …


SUBMISSION OF WRITTEN WORK

Page 1. IT U N IV ERSITY O F CO PEN H A G EN SUBMISSION OF WRITTEN WORK Class code: Name of course: Course manager: Course e-portfolio: Thesis or project title: Supervisor: Full Name: Birthdate (dd/mm-yyyy) …


Subword-based Compact Reconstruction of Word Embeddings

S Sasaki, J Suzuki, K Inui - Proceedings of the 2019 Conference of the North …, 2019

… or embedding vectors), especially those trained on a vast amount of text data, such as the Common Crawl (CC) cor … word embeddings trained from GloVe.840B and fastText.600B are available: https://github.com/losyer …


SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems

A Wang, Y Pruksachatkun, N Nangia, A Singh…

… We also include a baseline where for each task we simply predict the majority class, as well as a bag-of-words baseline where each input is represented as an average of its tokens' GloVe word vectors (300-dimensional and trained …


Supervised Multimodal Bitransformers for Classifying Images and Text

D Kiela, S Bhooshan, H Firooz, D Testuggine - arXiv preprint arXiv:1909.02950, 2019

… We describe each of the baselines in more detail below. • Bag of words (Bow) We sum 300-dimensional GloVe embeddings (Pennington, Socher, and Manning 2014) (trained on Common Crawl) for all words in the text …


Supplementary Material for “Multi-task Learning of Hierarchical Vision-Language Representation”

… Questions and captions were tokenized using Python Natural Language Toolkit (nltk) [2]. We used the vocabulary provided by the CommonCrawl-840B GloVe model for English word vectors [8], and set out-of-vocabulary words to unk …