Posts

The genesis of this blog post stems from a thoughtful discussion I had with my roommate. The impetus for this conversation originated from the recent widespread popularity of chatGPT, which utilizes transformer technology and, consequently, has put the attention mechanism once again in the spotlight. While there have been many discussions surrounding the attention mechanism, it is widely known that its structure comprises the “q-k-v” formulation - a fact that has largely been established since its inception in the article “Attention is all you need”. However, this prompts the question:‘where did it originate?’ This thought-provoking topic serves as the basis of this blog post.

Some understanding of Typora+Markdown.

Some understanding of computer vision basic tasks.

This blog talks about the historical developement of self-attention mechanism and the implementation of transformer decoder.

This blog talks about the historical developement of self-attention mechanism and the implementation of transformer decoder.

Wowchemy is designed to give technical content creators a seamless experience. You can focus on the content and Wowchemy handles the rest. Highlight your code snippets, take notes on math classes, and draw diagrams from textual representation.

Welcome 👋 We know that first impressions are important, so we’ve populated your new site with some initial content to help you get familiar with everything in no time.

Learn how to blog in Academic using Jupyter notebooks