늘보의 서랍

논문 링크 : Sequence to Sequence Learning with Neural Networks Deep Neural Networks (DNNs) are powerful models that have achieved excellent performance on difficult learning tasks. Although DNNs work well whenever large labeled training sets are available, they cannot be used to map sequences to sequences. In this pap arxiv.org 참고 자료 링크 : 1) 시퀀스-투-시퀀스(Sequence-to-Sequence, seq2seq) 이번 실습은 케라스 함수형 AP..

논문 링크 : Efficient Estimation of Word Representations in Vector Space We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previously best per arxiv.org 참고 자료 링크 : 02) 워드투벡터(Word2Vec) 앞서 원-핫 벡터는 단어 벡터 간 유의미한 유사도를 계산..

feedback-nn-train Explore and run machine learning code with Kaggle Notebooks | Using data from Feedback Prize - Evaluating Student Writing www.kaggle.com 참여했던 캐글 공모전의 1nd place solution을 분석해보려고 한다. 작성된 method 위주로 리뷰를 해보겠다. 교집합, 즉 얼마나 두 집단이 겹쳐있느냐를 파악하는 함수. set()을 활용했다. f1_score를 구하는 함수. log와 slient를 파라미터로 주어 slient에 따라 함수 아래의 문장을 실행하도록 하였다. 예측값과 실제값의 고유값을 비교해주고자 한듯 하다. slient는 오타인 것 같다. 그리고 아래에서..

Question answering - Hugging Face Course Time to look at question answering! This task comes in many flavors, but the one we’ll focus on in this section is called extractive question answering. This involves posing questions about a document and identifying the answers as spans of text in the d huggingface.co Hugging Face 코스 리뷰의 마지막, Question Answering Task. 여기서는 extractive question answering에 대..

Training a causal language model from scratch - Hugging Face Course Up until now, we’ve mostly been using pretrained models and fine-tuning them for new use cases by reusing the weights from pretraining. As we saw in Chapter 1, this is commonly referred to as transfer learning, and it’s a very successful strategy for a huggingface.co Causal language model을 처음부터 학습시켜보는 강의 내용. 여기서 Text generation ..

Main NLP tasks - Hugging Face Course In this section we’ll take a look at how Transformer models can be used to condense long documents into summaries, a task known as text summarization. This is one of the most challenging NLP tasks as it requires a range of abilities, such as understandin huggingface.co 문서를 요약하는 text summarization에 대해 알아보자. 필요한 데이터를 load하고 랜덤 샘플을 뽑아 출력해보았다. English와 Spanish의 b..

2nd Place Solution - [CV741 Public727 Private740] Explore and run machine learning code with Kaggle Notebooks | Using data from multiple data sources www.kaggle.com 최근에 참여했던 캐글 공모전의 2nd place를 차지한 코드를 리뷰하고자 한다. 공모전 참여 기간 동안 노트북 작성자의 코드를 보며 많이 배웠기 때문에 최종 코드 또한 살펴보고 싶었다. 코드를 직접 돌려볼 수 있는 환경이 되지 않아서 코드를 읽고 분석하는 작업만 수행하고자 한다. Inference Script with Post Process The following Python script accepts a fi..

Main NLP tasks - Hugging Face Course Let’s now dive into translation. This is another sequence-to-sequence task, which means it’s a problem that can be formulated as going from one sequence to another. In that sense the problem is pretty close to summarization, and you could adapt what we huggingface.co sequence-to-sequence task인 translation에 대해 학습해보자. task에 맞는 dataset을 불러오고 그 안의 샘플을 뽑아보았다. pipe..

Main NLP tasks - Hugging Face Course For many NLP applications involving Transformer models, you can simply take a pretrained model from the Hugging Face Hub and fine-tune it directly on your data for the task at hand. Provided that the corpus used for pretraining is not too different from th huggingface.co This process of fine-tuning a pretrained language model on in-domain data is usually call..

Main NLP tasks - Hugging Face Course The first application we’ll explore is token classification. This generic task encompasses any problem that can be formulated as “attributing a label to each token in a sentence,” such as: Of course, there are many other types of token classification huggingface.co 이전에 Tokenizer에 대해 학습하였는데 이제는 그것을 활용해서 토큰을 분류해주는 Token classification에 대해 학습해보고자 한다. 여기서 소개하는 것은..