New Show Hacker News story: Show HN: A surprisingly effective way to predict token importance in LLM prompts

Show HN: A surprisingly effective way to predict token importance in LLM prompts
11 by shayanjm | 3 comments on Hacker News.
We explored a novel method to gauge the significance of tokens in prompts given to large language models, without needing direct model access. Essentially, we just did an ablation study on the prompt using cosine similarity of the embeddings as the measure. We got surprisingly promising results when comparing this really simple approach to integrated gradients. Curious to hear thoughts from the community!

Comments

Post a Comment

Popular posts from this blog

New Show Hacker News story: Show HN: A local Python prototyping tool for Jupyter and Streamlit

Show HN: A local Python prototyping tool for Jupyter and Streamlit 4 by galenmarchetti | 0 comments on Hacker News. Hey HN! I built a local Python prototyping tool that is finally the Python development environment I've always wanted. It has a Jupyter notebook for data crunching, a database of your choice (Python or MongoDB), and a Streamlit app for building a frontend visualization. You can edit the Streamlit backend via an embedded VSCode editor, or locally on your own IDE. The best part for me is that the database connectors within Jupyter and Streamlit are configured out-of-the-box, so you don't need to spend time thinking about how to tie all that together - you can just pick the database you want to use and get going. Disclaimer: I do also work on the tool that deploys all this under the hood, but this project is a personal hackweek project that I threw together so I could develop Python apps on my own
Read more

New Show Hacker News story: Show HN: Natural language Twitter search using Codex

Show HN: Natural language Twitter search using Codex 3 by andyk | 0 comments on Hacker News. We built a structured search engine for Twitter called Bird SQL, available at https://ift.tt/7BU0hHb . Our search interface uses OpenAI Codex to translate natural language to SQL. Our backend then verifies the SQL, executes it, and displays the results on the web app. This makes large structured datasets like a scrape of Twitter easy for anyone to explore. As background, while working on text-to-SQL as a general problem, we came to believe one of its most powerful applications is as a search tool because: - SQL is hard to write by hand and prone to errors - It allows you to iterate quickly if you’re exploring a new dataset - A lot of contextual information that you’d normally have to internalize (e.g. your data’s schema) can be automatically generated and offloaded to the language model Using large language models (LLMs) like Codex to write the SQL for you means you don’t have to worry about...
Read more