# One-shot action selection
The plan:
1. [Choose what action we’d ideally execute in a context](#choosing-actions)
2. [Actually execute the action that was chosen](#executing-actions)
## Choosing actions
Let’s start by just making a prompt for choosing an action, without hooking it up to actually executing the action. This way, we can check whether the model is even capable of making reasonable choices.
There’s a long list of actions we could choose between. For this first version, we’ll limit ourselves to:
1. Web search
2. Computation
3. Reasoning
### **Representing actions**
Let’s first represent the actions as a data type. For each action we’ll also store an associated description that will help the model choose between them, and the recipe that runs the action:
{% code title="answer\_by\_dispatch/types.py" overflow="wrap" %}
```python
from dataclasses import dataclass
from typing import Protocol
from ice.recipes.primer.answer_by_computation import answer_by_computation
from ice.recipes.primer.answer_by_reasoning import answer_by_reasoning
from ice.recipes.primer.answer_by_search import answer_by_search
class QuestionRecipe(Protocol):
async def __call__(self, question: str) -> str:
...
@dataclass
class Action:
name: str
description: str
recipe: QuestionRecipe
action_types = [
Action(
name="Web search",
description="Run a web search using Google. This is helpful if the question depends on obscure facts or current information, such as the weather or the latest news.",
recipe=answer_by_search,
),
Action(
name="Computation",
description="Run a computation in Python. This is helpful if the question depends on calculation or other mechanical processes that you can specify in a short program.",
recipe=answer_by_computation,
),
Action(
name="Reasoning",
description="Write out the reasoning steps. This is helpful if the question involves logical deduction or evidence-based reasoning.",
recipe=answer_by_reasoning,
),
]
```
{% endcode %}
### **From actions to prompts**
We render the actions as an action selection prompt like this:
{% code title="answer\_by\_dispatch/prompt.py" overflow="wrap" %}
```python
from fvalues import F
from ice.recipes.primer.answer_by_dispatch.types import *
def make_action_selection_prompt(question: str) -> str:
action_types_str = F("\n").join(
[
F(f"{i+1}. {action_type.description}")
for i, action_type in enumerate(action_types)
]
)
return F(
f"""You want to answer the question "{question}".
You have the following options:
{action_types_str}
Q: Which of these options do you want to use before you answer the question? Choose the option that will most help you give an accurate answer.
A: I want to use option #"""
).strip()
```
{% endcode %}
So, `make_action_selection_prompt("How many people live in Germany?")` results in:
{% code overflow="wrap" %}
```
You want to answer the question "How many people live in Germany?".
You have the following options:
1. Run a web search using Google. This is helpful if the question depends on obscure facts or current information, such as the weather or the latest news.
2. Run a computation in Python. This is helpful if the question depends on calculation or other mechanical processes that you can specify in a short program.
3. Write out the reasoning steps. This is helpful if the question involves logical deduction or evidence-based reasoning.
Q: Which of these options do you want to use before you answer the question? Choose the option that will most help you give an accurate answer.
A: I want to use option #
```
{% endcode %}
### **Choosing the right action**
We’ll treat action choice as a classification task, and print out the probability of each action:
{% code title="answer\_by\_dispatch/classify.py" %}
```python
from ice.recipe import recipe
from ice.recipes.primer.answer_by_dispatch.prompt import *
async def answer_by_dispatch(question: str = "How many people live in Germany?"):
prompt = make_action_selection_prompt(question)
choices = tuple(str(i) for i in range(1, 6))
probs, _ = await recipe.agent().classify(prompt=prompt, choices=choices)
return list(zip(probs.items(), [a.name for a in action_types]))
recipe.main(answer_by_dispatch)
```
{% endcode %}
Let’s test it:
{% code overflow="wrap" %}
```shell
python answer_by_dispatch/classify.py --question "How many people live in Germany?"
```
{% endcode %}
```python
[
(('1', 0.9887763823328329), 'Web search'),
(('2', 0.0004275099864492222), 'Computation'),
(('3', 0.010796107680717818), 'Reasoning')
]
```
Web search seems like the correct solution here.
```shell
python answer_by_dispatch/classify.py --question "What is sqrt(2^8)?"
```
```python
[
(('1', 5.399225731055571e-06), 'Web search'),
(('2', 0.9998907431467178), 'Computation'),
(('3', 0.00010382736418811754), 'Reasoning')
]
```
Clearly a computation question.
{% code overflow="wrap" %}
```shell
python answer_by_dispatch/classify.py --question "Is transhumanism desirable?"
```
{% endcode %}
```python
[
(('1', 0.00014451187131909118), 'Web search'),
(('2', 0.006603133965933587), 'Computation'),
(('3', 0.9932523541627474), 'Reasoning')
]
```
Reasoning makes sense here.
{% code overflow="wrap" %}
```shell
python answer_by_dispatch/classify.py --question "What are the effects of climate change?"
```
{% endcode %}
```python
[
(('1', 0.7689459560875318), 'Web search'),
(('2', 0.008138518726847932), 'Computation'),
(('3', 0.22291552518562033), 'Reasoning')
]
```
Mostly a web search question, but might need some clarification.
### Executing actions
Now let’s combine the action selector with the chapters on web search, computation, and reasoning to get a single agent that can choose the appropriate action.
This is extremely straightforward—since all the actions are already associated with subrecipes, all we need to do is run the chosen subrecipe:
{% code title="answer\_by\_dispatch/execute.py" %}
```python
from ice.recipe import recipe
from ice.recipes.primer.answer_by_dispatch.prompt import *
async def select_action(question: str) -> Action:
prompt = make_action_selection_prompt(question)
choices = tuple(str(i) for i in range(1, 6))
choice_probs, _ = await recipe.agent().classify(prompt=prompt, choices=choices)
best_choice = max(choice_probs.items(), key=lambda x: x[1])[0]
return action_types[int(best_choice) - 1]
async def answer_by_dispatch(question: str = "How many people live in Germany?") -> str:
action = await select_action(question)
result = await action.recipe(question=question)
return result
recipe.main(answer_by_dispatch)
```
{% endcode %}
Let’s try it with our examples above:
{% code overflow="wrap" %}
```
$ python answer_by_dispatch/execute.py --question "How many people live in Germany?"
The current population of Germany is 84,370,487 as of Monday, September 12, 2022, based on Worldometer elaboration of the latest United Nations data.
```
{% endcode %}
```
$ python answer_by_dispatch/execute.py --question "What is sqrt(2^8)?"
16.0
```
{% code overflow="wrap" %}
```
$ python answer_by_dispatch/execute.py --question "Is transhumanism desirable?"
It is up to each individual to decide whether or not they believe transhumanism is desirable.
```
{% endcode %}
These are arguably better answers than we’d get without augmentation.
### Exercises
1. Suppose that actions are taking place within the context of a long document. Add an action type for searching for a particular phrase in the document and returning the results.
2. Add an action type for debate:
{% code overflow="wrap" %}
```python
Action(
name="Debate",
description='Run a debate. This is helpful if the question is a pro/con question that involves involves different perspectives, arguments, and evidence, such as "Should marijuana be legalized?" or "Is veganism better for the environment?".'
)
```
{% endcode %}
Get feedback on exercise solutions
If you want feedback on your exercise solutions, submit them through [this form](https://docs.google.com/forms/d/e/1FAIpQLSdNNHeQAT7GIzn4tdsVYCkrVEPMNaZmBFkZCAJdvTvLzUAnzQ/viewform). We—the team at Ought—are happy to give our quick take on whether you missed any interesting ideas.
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://primer.ought.org/chapters/action-selection/one-shot-action-selection.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.