Comments (1)
Hey, sorry for the late reply. The prompts are all just few-shot demonstrations plus the specialized prompting listed in the prompt Figure of the paper.
You can download all the testsets from this link. In each json file (corresponding to each QA dataset), there's a list of demos, we basically concatenate those demonstration examples as the prompt.
So for example, if you want to use the multihop expert, then each demonstration should be the question + cot rationale + "Therefore, the final answer is xxx."
Then you just append your test question to get the CoT rationale and answer prediction. It's the exact same thing for the math expert, except that we use demo examples from GSM8K rather than HotpotQA, and the explanations (CoT) for each question are from the original dataset.
For the factual expert, the demonstration examples are just the question + answer
from NQ, but for each test question, we also prepend the top retrieved passages before the question (you can check the uniqa_predictions_final logs to see the exact formatting. And it's the same thing for the commonsense expert, except that the "passages" are generated by GPT itself rather than retrieved.
I know it sounds a bit confusing, but I'm happy to jump onto a Zoom call if you want to replicate our experiment setup and have additional questions.
from more.
Related Issues (1)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from more.