Junecompete
— Travel, Learning, Projects — 3 min read
Projects
I didn't attempt many projects because I wanted to learn some fundamentals first, I had some travels planned, and I bought TOTK.
PodcastGPT
The top 5 papers of a computer science category from Arxiv are sent to ChatGPT and asked to translate into a podcast. Originally I broke the paper up into sections and fed them into ChatGPT api one at a time due to small context size, but the release of gpt3.5-16k let me send in almost the entire paper. On average I now send 2.5 chunks per paper (12,000 tokens each). Advertisements are also generated by ChatGPT (I need some way to fund this) and any reference to real companies are accidental.
Voices are from the free google translate api (gtts) because the much better, high quality api ElevenLabs costs too much for me to sponsor this.
Music is from Google's new MusicLM using the prompt Jazz with Dubstep Saxophone.
You can view the code here.
The run function is ran with 5 Arxiv CS categories I found interesting and are subject to change if I feel like it.
The schedule of these runs is managed by Github Actions, which can be used as a free cronjob server with decent execution time for small IO bound apps like this.
Learning
Math for Machine Learning
I watched a combination of coursera classes and youtube videos (as well as talking to ChatGPT) to learn the basics of the math behind machine learning. I wanted to have an intuitive grasp of the subject, so that when I build or interact with models, its no longer a black box.
Coursera Math classes
Linear Algebra The homework was much harder than the videos. However I already learned these topics around 8 years ago, just a matter of learning the formulas again. Surprisingly, the quadratic formula was the hardest to remember.
3Blue1Brown Linear Algebra
These help visualize vectors and linear transformations in a nice way. 3B1B also has many other nice math videos, especially for multi-variate calculus.
Machine Learning
Karpathy's MicroGrad
This video taught me how the nuerons of a nueral network compute their forward and backward passes intuitively.
Jeremy Howard (fast.ai) Stable Foundations
Amazing teacher, he spends 10-15 lectures going from the very basics of how nueral networks and optimizers work all the way into creating a diffusion network for images.
Unlike other shorter tutorials, he refuses to let you use a wrapper for something (except torch.Tensor) without first implementing it.
Other
Principles of Microeconomics
I'm only halfway through this, but it is very fascinating. He goes from intuitive ideas of economics, to their math representation, and back to the intuitive idea. I'm hoping to finish this in July.
The Human Brain
Really interesting overview of the brain and how scientists run experiments. Essentially interpretibility of our internal DNN without having a good debugger. Again I'm only halfway through, but hope to finish soon.
