Hello I’m Jess and today I’m gonna show you how to train your own GPT instance on the Google Quality Rater Guidelines and also tell you why it may not be a good idea to do that.

Last week I made a thread about this on the blue hell bird site and now this is a longer version of that Tweet Thread.

Copy the Colaboratory to follow along at home!

Introduction

Explanation of GPT and its capabilities

GPT is the hottest topic in that it seems everyone is talking about it. I’m talking about it, Andrew’s talked about it, every SEO blog has talked about it.

If you are miraculously inoculated against hearing about GPT, this ends now.

GPT (Generative Pre-trained Transformer) is a Large Language Model. This means it uses NLP (natural language processing, computer to text to computer) to understand language and it’s big. Huge, even. If you’re not a huge dweeb, like I am, that’s all you need to know. Unfortunately I was cursed by a hag behind a Dunkin Donuts with knowledge and the inability to stop talking, so let’s get a little deeper into all this stuff.

GPT uses a type of model called a transformer, which is a neural network that can learn context and meaning by tracking relationships between words. This owns! Previously it was very hard to do this in such a complicated manner.

Transformers were only really cooking in 2017, when Google published a paper called “attention is all you need.” Around the same time, ULMFiT (which is an…