Skip to content

Commit 27eb38b

Browse files
committed
Convert CoT and plan instructors to simple_task for more parallelism.
1 parent 955d9de commit 27eb38b

File tree

7 files changed

+137
-129
lines changed

7 files changed

+137
-129
lines changed

airoboros/instructors/cot.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
from airoboros.instructors.inline_qa import generate as generate_inline
1+
from airoboros.instructors.simple_task import generate as generate_simple
22

33

44
async def generate(instructor):
55
"""Generator for chain-of-thought training data."""
6-
async for item in generate_inline(instructor, "cot"):
6+
async for item in generate_simple(instructor, "cot", filter_response=False):
77
yield item

airoboros/instructors/plan.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
from airoboros.instructors.inline_qa import generate as generate_inline
1+
from airoboros.instructors.simple_task import generate as generate_simple
22

33

44
async def generate(instructor):
55
"""Generator for rewoo style planning."""
6-
async for item in generate_inline(instructor, "plan", filter_response=False):
6+
async for item in generate_simple(instructor, "plan", filter_response=False):
77
yield item

airoboros/instructors/prompts/cot.txt

Lines changed: 9 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -1,88 +1,22 @@
1-
I would like your help in producing a chain-of-thought style question and response.
1+
I would like your help in producing a chain-of-thought style questions/instructions.
22

33
Below are a few examples:
44

5-
Example 1:
6-
QUESTION: Jackie has 3 brothers. Each brother has 2 sisters. How many sisters does Jackie have? Give several possible answers to the question, ensuring you provide step-by-step reasoning for each. After you have provided possible solutions, rank the solutions in order of accuracy and completeness, then select the best possible output based on available information.
5+
Example 1: Jackie has 3 brothers. Each brother has 2 sisters. How many sisters does Jackie have? Give several possible answers to the question, ensuring you provide step-by-step reasoning for each. After you have provided possible solutions, rank the solutions in order of accuracy and completeness, then select the best possible output based on available information.
76

8-
ANSWER:
9-
Solution 1:
10-
The information provided states that "Each brother has 2 sisters." This includes Jackie in the count because she is a sister to her brothers. Considering there are three brothers and each of them has the same two sisters, we can deduce that the siblings consist of 3 brothers and 2 sisters in total. If we remove Jackie from the sibling count (since we are trying to find out how many sisters Jackie herself has), we are left with one other girl in the sibling group.
11-
12-
Final answer: Jackie has 1 sister.
13-
14-
Solution 2:
15-
In this scenario, we can assume a complex family structure, such as half-siblings or step-siblings. "Each brother has 2 sisters" could imply that there are sisters not related to Jackie, like half-sisters or step-sisters. Given this complexity, there could be a situation where Jackie doesn't have any sisters at all.
16-
17-
Final answer: Jackie has 0 sisters.
18-
19-
Solution 3:
20-
The situation could involve a non-traditional family structure where each brother has two sisters, but they are not the same for every brother. It's possible that Jackie is not considered a sister by one or more of her brothers due to reasons like estrangement or legal technicalities. In this case, the number of sisters that Jackie has could be different from the number her brothers have.
21-
22-
Final answer: This would depend on the specifics of the family dynamics, and thus can't be definitively answered.
23-
24-
Ranking of solutions from best to worst:
25-
1. Solution 1: This answer is based on a straightforward interpretation of the provided information and the most common family structures. It uses simple deductive reasoning and doesn't require making additional assumptions.
26-
2. Solution 2: While this scenario is possible, it makes assumptions about more complex family structures, which might not apply in this case. The provided information doesn't suggest any complexity in the family structure.
27-
3. Solution 3: This answer is the most speculative as it relies on assumptions about estrangement or legal technicalities. The likelihood of these circumstances is not suggested by the provided information, so this solution seems the least probable.
28-
29-
Best and final answer: Jackie has 1 sister.
30-
31-
32-
Example 2:
33-
QUESTION: It take 4 hours to dry 5 shirts out in the sun. How long would it take to dry 20 shirts? Use chain-of-thought reasoning to generate several possible responses, then select the best response based on all available information, intuition, and likelihood of correctness.
34-
35-
ANSWER:
36-
Solution 1:
37-
Consider the time of day, and how much sunlight is remaining. For instance, if you have only 4 hours of sunlight left in the day, you would not be able to dry all 20 shirts within the day, if you can only dry 5 shirts at a time. Assuming the following day has at least 12 hours of sunlight and conditions are sufficient for the 5 shirts to dry in 4 hours consistently, we can write it as:
38-
total time = dry time per batch size * number of batches + time delayed due to lack of sunlight
39-
40-
In this case, the dry time per batch of 5 shirts is 4 hours, and the number of batches is (20 / 5 = 4).
41-
42-
Since we make an assumption that we have 12 hours of drying time, that implies we have a delay of (24 hours in a day - 12 hours = 12 hours) of delay time.
43-
44-
The total amount of time is therefore:
45-
4 * 4 + 12 = 28 hours.
46-
47-
Final answer: It would take 28 hours to dry 20 shirts, assuming 12 hours of sufficient weather and solar conditions with a batch size of 5 shirts.
48-
49-
Solution 2:
50-
It is given that it takes 4 hours to dry 5 shirts.
51-
52-
This means that 1 shirt would take the same 4 hours to dry, because the task is parallelizable.
53-
54-
Since each shirt dries individually in parallel, the drying time doesn't stack. This means that it doesn't matter how many shirts we're drying at once, as long as there's enough space for all shirts to be exposed to the environment equally, they will all take 4 hours to dry.
55-
56-
So, it would still take 4 hours to dry 20 shirts under the assumption that they're all drying in parallel, given that they're exposed to similar conditions as when drying the initial 5 shirts.
57-
58-
Final answer: It would still take 4 hours to dry 20 shirts, since the task is parallelizable.
59-
60-
Ranking of solutions from best to worst:
61-
1. Solution 2: This answer is most likely correct because it uses straightforward reasoning based on the information provided, which does not indicate that space or sunlight availability is a limiting factor.
62-
2. Solution 1: This answer is less likely, considering the task is most likely parallelizable, and we are making several assumptions, including the amount of daylight remaining, amount of time per day in which shirts dry in exactly 4 hours.
63-
64-
Best and final answer: It would still take 4 hours to dry 20 shirts, since the task is parallelizable.
65-
66-
End of examples.
67-
68-
The possible solutions should always have the reasoning first, then the final answer. Don't ever put the final answer first, then reasoning.
69-
70-
Be sure the new questions contain instructions to use chain-of-thought reasoning, as the examples do.
7+
Example 2: It take 4 hours to dry 5 shirts out in the sun. How long would it take to dry 20 shirts? Use chain-of-thought reasoning to generate several possible responses, then select the best response based on all available information, intuition, and likelihood of correctness.
718

729
Provide a set of {batch_size} new, complex, unique, diverse tasks.
7310

74-
Make sure you fully understand each solution before providing a ranking. The position of the solution does not always correspond to it's ranking, i.e. sometimes solution 2 or 3 can be better that solution 1, and therefore the ranking should reflect that. Always rank the solutions based on accuracy, completeness, and probability of being correct, not based on their position in the list of possible solutions.
11+
{topic_avoidance}
7512

76-
Be sure to include at least 2, preferably 3 possible solutions for each question.
13+
Be sure to include a wide variety of tasks, including tasks that explore ideas of set theory, information theory, parallelism, logic, extrapolation from scientific studies, etc., but also be sure to only include questions that have multiple potentially correct answers.
7714

78-
All output text should be in {language}, but the exact terms "QUESTION" and "ANSWER" are special tokens that must not be translated.
15+
All output text should be in {language}, but the exact term "TSK" is a special token that must not be translated.
7916

8017
The output format should be:
81-
QUESTION: [question 1]
82-
ANSWER: [answer to question 1]
83-
84-
QUESTION: [question 2]
85-
ANSWER: [answer to question 2]
18+
TSK 1. [question 1, including instructions to provide multiple solutions, rank, and select best]
19+
TSK 2. [question 2, including instructions to provide multiple solutions, rank, and select best]
8620
...
8721

88-
Be sure to use this precise format, i.e. don't use "QUESTION 1: [question]", other otherwise manupulate the labels.
22+
Be sure to include "TSK", untranslated, as a prefix as described in response format.
Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
Please provide a chain-of-thought style answer to the provide input.
2+
3+
Below are a few exampless of question/response pairs.
4+
5+
Example 1:
6+
Input: Jackie has 3 brothers. Each brother has 2 sisters. How many sisters does Jackie have? Give several possible answers to the question, ensuring you provide step-by-step reasoning for each. After you have provided possible solutions, rank the solutions in order of accuracy and completeness, then select the best possible output based on available information.
7+
8+
Example 1 response:
9+
Solution 1:
10+
The information provided states that "Each brother has 2 sisters." This includes Jackie in the count because she is a sister to her brothers. Considering there are three brothers and each of them has the same two sisters, we can deduce that the siblings consist of 3 brothers and 2 sisters in total. If we remove Jackie from the sibling count (since we are trying to find out how many sisters Jackie herself has), we are left with one other girl in the sibling group.
11+
12+
Final answer: Jackie has 1 sister.
13+
14+
Solution 2:
15+
In this scenario, we can assume a complex family structure, such as half-siblings or step-siblings. "Each brother has 2 sisters" could imply that there are sisters not related to Jackie, like half-sisters or step-sisters. Given this complexity, there could be a situation where Jackie doesn't have any sisters at all.
16+
17+
Final answer: Jackie has 0 sisters.
18+
19+
Solution 3:
20+
The situation could involve a non-traditional family structure where each brother has two sisters, but they are not the same for every brother. It's possible that Jackie is not considered a sister by one or more of her brothers due to reasons like estrangement or legal technicalities. In this case, the number of sisters that Jackie has could be different from the number her brothers have.
21+
22+
Final answer: This would depend on the specifics of the family dynamics, and thus can't be definitively answered.
23+
24+
Ranking of solutions from best to worst:
25+
1. Solution 1: This answer is based on a straightforward interpretation of the provided information and the most common family structures. It uses simple deductive reasoning and doesn't require making additional assumptions.
26+
2. Solution 2: While this scenario is possible, it makes assumptions about more complex family structures, which might not apply in this case. The provided information doesn't suggest any complexity in the family structure.
27+
3. Solution 3: This answer is the most speculative as it relies on assumptions about estrangement or legal technicalities. The likelihood of these circumstances is not suggested by the provided information, so this solution seems the least probable.
28+
29+
Best and final answer: Jackie has 1 sister.
30+
31+
32+
Example 2:
33+
Input: It take 4 hours to dry 5 shirts out in the sun. How long would it take to dry 20 shirts? Use chain-of-thought reasoning to generate several possible responses, then select the best response based on all available information, intuition, and likelihood of correctness.
34+
35+
Response to example 2:
36+
Solution 1:
37+
Consider the time of day, and how much sunlight is remaining. For instance, if you have only 4 hours of sunlight left in the day, you would not be able to dry all 20 shirts within the day, if you can only dry 5 shirts at a time. Assuming the following day has at least 12 hours of sunlight and conditions are sufficient for the 5 shirts to dry in 4 hours consistently, we can write it as:
38+
total time = dry time per batch size * number of batches + time delayed due to lack of sunlight
39+
40+
In this case, the dry time per batch of 5 shirts is 4 hours, and the number of batches is (20 / 5 = 4).
41+
42+
Since we make an assumption that we have 12 hours of drying time, that implies we have a delay of (24 hours in a day - 12 hours = 12 hours) of delay time.
43+
44+
The total amount of time is therefore:
45+
4 * 4 + 12 = 28 hours.
46+
47+
Final answer: It would take 28 hours to dry 20 shirts, assuming 12 hours of sufficient weather and solar conditions with a batch size of 5 shirts.
48+
49+
Solution 2:
50+
It is given that it takes 4 hours to dry 5 shirts.
51+
52+
This means that 1 shirt would take the same 4 hours to dry, because the task is parallelizable.
53+
54+
Since each shirt dries individually in parallel, the drying time doesn't stack. This means that it doesn't matter how many shirts we're drying at once, as long as there's enough space for all shirts to be exposed to the environment equally, they will all take 4 hours to dry.
55+
56+
So, it would still take 4 hours to dry 20 shirts under the assumption that they're all drying in parallel, given that they're exposed to similar conditions as when drying the initial 5 shirts.
57+
58+
Final answer: It would still take 4 hours to dry 20 shirts, since the task is parallelizable.
59+
60+
Ranking of solutions from best to worst:
61+
1. Solution 2: This answer is most likely correct because it uses straightforward reasoning based on the information provided, which does not indicate that space or sunlight availability is a limiting factor.
62+
2. Solution 1: This answer is less likely, considering the task is most likely parallelizable, and we are making several assumptions, including the amount of daylight remaining, amount of time per day in which shirts dry in exactly 4 hours.
63+
64+
Best and final answer: It would still take 4 hours to dry 20 shirts, since the task is parallelizable.
65+
66+
67+
End of examples.
68+
69+
The possible solutions should always have the reasoning first, then the final answer. Don't ever put the final answer first, then reasoning.
70+
71+
Make sure you fully understand each solution before providing a ranking. The position of the solution does not always correspond to it's ranking, i.e. sometimes solution 2 or 3 can be better that solution 1, and therefore the ranking should reflect that. Always rank the solutions based on accuracy, completeness, and probability of being correct, not based on their position in the list of possible solutions.
72+
73+
Be sure to include at least 2, preferably 3 possible solutions.
74+
75+
All output text should be in {language}.
76+
77+
Input: {instruction}

0 commit comments

Comments
 (0)