늘보의 서랍

github의 fork 기능을 cli에서 하고자 생각해봤다. 하지만 git bash cli에는 해당 명령어가 없는 것 같고 방법이라면 git remote add upstream으로 레포 연결하고 양쪽을 왔다갔다 할 수밖에 없는 듯 싶다... 이게 fork인가? 그래서 cli에서 하려면 github cli를 환경에 설치해줘야 하는데 그것은 GitHub - cli/cli: GitHub’s official command line tool GitHub’s official command line tool. Contribute to cli/cli development by creating an account on GitHub. github.com 에서 처럼 MacOs는 brew install gh / Conda는 ..

re — 정규식 연산 — Python 3.10.4 문서 re — 정규식 연산 소스 코드: Lib/re.py 이 모듈은 Perl에 있는 것과 유사한 정규식 일치 연산을 제공합니다. 패턴과 검색 할 문자열은 모두 유니코드 문자열(str)과 8비트 문자열(bytes)이 될 수 있습니 docs.python.org re — Regular expression operations — Python 3.10.4 documentation re — Regular expression operations Source code: Lib/re.py This module provides regular expression matching operations similar to those found in Perl. Both patter..

정규식 HOWTO — Python 3.10.4 문서 대소 문자를 구분하지 않는 일치를 수행합니다; 문자 클래스와 리터럴 문자열은 대소 문자를 무시하여 문자와 일치합니다. 예를 들어 [A-Z]는 소문자와도 일치합니다. ASCII 플래그로 ASCII가 아닌 docs.python.org Regular Expression HOWTO — Python 3.10.4 documentation Perform case-insensitive matching; character class and literal strings will match letters by ignoring case. For example, [A-Z] will match lowercase letters, too. Full Unicode matching a..

SELECT * FROM 테이블명; DESC 테이블명; SELECT 열1, 열2 FROM 테이블명 WHERE 조건식; 값이 서로 다른 경우, '' ( WHERE no 2;) NULL 값을 검색할 경우, IS NULL (WHERE birthday IS NULL;) =, , >, >=,

Question answering - Hugging Face Course Time to look at question answering! This task comes in many flavors, but the one we’ll focus on in this section is called extractive question answering. This involves posing questions about a document and identifying the answers as spans of text in the d huggingface.co Hugging Face 코스 리뷰의 마지막, Question Answering Task. 여기서는 extractive question answering에 대..

Training a causal language model from scratch - Hugging Face Course Up until now, we’ve mostly been using pretrained models and fine-tuning them for new use cases by reusing the weights from pretraining. As we saw in Chapter 1, this is commonly referred to as transfer learning, and it’s a very successful strategy for a huggingface.co Causal language model을 처음부터 학습시켜보는 강의 내용. 여기서 Text generation ..

Main NLP tasks - Hugging Face Course In this section we’ll take a look at how Transformer models can be used to condense long documents into summaries, a task known as text summarization. This is one of the most challenging NLP tasks as it requires a range of abilities, such as understandin huggingface.co 문서를 요약하는 text summarization에 대해 알아보자. 필요한 데이터를 load하고 랜덤 샘플을 뽑아 출력해보았다. English와 Spanish의 b..

Main NLP tasks - Hugging Face Course Let’s now dive into translation. This is another sequence-to-sequence task, which means it’s a problem that can be formulated as going from one sequence to another. In that sense the problem is pretty close to summarization, and you could adapt what we huggingface.co sequence-to-sequence task인 translation에 대해 학습해보자. task에 맞는 dataset을 불러오고 그 안의 샘플을 뽑아보았다. pipe..

Main NLP tasks - Hugging Face Course For many NLP applications involving Transformer models, you can simply take a pretrained model from the Hugging Face Hub and fine-tune it directly on your data for the task at hand. Provided that the corpus used for pretraining is not too different from th huggingface.co This process of fine-tuning a pretrained language model on in-domain data is usually call..

Main NLP tasks - Hugging Face Course The first application we’ll explore is token classification. This generic task encompasses any problem that can be formulated as “attributing a label to each token in a sentence,” such as: Of course, there are many other types of token classification huggingface.co 이전에 Tokenizer에 대해 학습하였는데 이제는 그것을 활용해서 토큰을 분류해주는 Token classification에 대해 학습해보고자 한다. 여기서 소개하는 것은..