diff --git a/lepton/starterkits/images/chat-data-curation-pipeline.png b/lepton/starterkits/images/chat-data-curation-pipeline.png new file mode 100644 index 0000000..759f732 Binary files /dev/null and b/lepton/starterkits/images/chat-data-curation-pipeline.png differ diff --git a/lepton/starterkits/synthetic-qa-data-gen-with-nemotron.ipynb b/lepton/starterkits/synthetic-qa-data-gen-with-nemotron.ipynb new file mode 100644 index 0000000..f7a2836 --- /dev/null +++ b/lepton/starterkits/synthetic-qa-data-gen-with-nemotron.ipynb @@ -0,0 +1,775 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "cde135f2", + "metadata": {}, + "source": [ + "# Data Curation with NIMs: Synthetic Chat QA Generation\n", + "\n", + "For chat data, prompts can come from both public, real-world user interactions and a synthetic data generation pipeline. Synthetic prompts cover various tasks such as open QA, closed QA, and creative writing. \n", + "\n", + "For each prompt task, the LLM can be seeded for generation with a diverse set of topics or keywords so that the prompts cover a wide variety of topics. For the responses, LLMs can be prompted for multiple generations in order to do rejection sampling with a reward model and generate higher quality synthetic data.\n" + ] + }, + { + "cell_type": "markdown", + "id": "7db5817a", + "metadata": {}, + "source": [ + "### Requirements:\n", + "* Container: `nvcr.io/nvidia/nemo:25.07`\n", + "* GPUs: One NVIDIA GPU with at least 20 GB of GPU memory is required for Endpoint deployment.\n", + "* Storage: To persist datasets, mount shared storage in the Dev Pod.\n", + "\n", + "---\n", + "\n", + "**Table of Contents**\n", + "\n", + "* API Key Setup\n", + "* OpenAI Client\n", + "* Prompt Templates\n", + " * Topics & Subtopics\n", + " * Generating Questions\n", + " * Generating Responses to Questions\n", + "* Summary & Next Steps\n", + "\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "96ed21a2", + "metadata": {}, + "source": [ + "![Figure 1. Chat data curation pipeline](images/chat-data-curation-pipeline.png)\n" + ] + }, + { + "cell_type": "markdown", + "id": "543142a2", + "metadata": {}, + "source": [ + "The following notebook will demonstrate how to leverage [Nemotron-nano-9b-v2](https://build.nvidia.com/nvidia/nvidia-nemotron-nano-9b-v2) through build.nvidia.com to synthetically generate chat QA data.\n", + "The pipeline is designed to create a preference dataset suitable for training a reward model using the [SteerLM method](https://docs.nvidia.com/nemo-framework/user-guide/latest/modelalignment/steerlm.html), which can be used for training an RLHF Reward Model or for DPO with a framework, such as [NeMo-RL](https://github.com/NVIDIA-NeMo/RL)." + ] + }, + { + "cell_type": "markdown", + "id": "73c59386", + "metadata": {}, + "source": [ + "### Inference Endpoint Setup\n", + "In this notebook, we will launch an inference server using DGX Cloud Lepton's Endpoints to be able to send prompts to a model. The notebook walks through this process, but you can also use `build.nvidia.com` to send an API request to an existing NVIDIA-managed endpoint - more on that later on.\n", + "\n", + "First, we need to setup an Endpoint to host the model we will use for generating datasets. The cell below configures several environment settings required for accessing DGX Cloud Lepton. You will need an API key for accessing your workspace. Follow [this link](https://docs.nvidia.com/dgx-cloud/lepton/features/workspace/token/) to generate an API key for authenticating with your cluster. Once generated, add it to the `LEPTON_KEY` output in the format `:` (without brackets) where `` is the unique identifier for your workspace.\n", + "\n", + "Next, update the `RESOURCE_SHAPE` and `NODE_GROUP` variables to match your workspace. The default model only needs approximately 20 GB of GPU memory so select an appropriate GPU resource for deploying the model." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "37b79cc5-0297-4668-b2a0-e9f2aef11af0", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import re\n", + "import subprocess\n", + "import time\n", + "\n", + "BASE_MODEL = \"nvidia/nvidia-nemotron-nano-9b-v2\" # Optionally use a different model for requests\n", + "ACCESS_TOKEN = \"my-access-token\" # Set the password for authenticating endpoint requests\n", + "SAVE_DIRECTORY = \"\" # Specify the absolute path to save the generated data. To save on shared storage, must be a mounted storage path\n", + "\n", + "os.environ[\"LEPTON_KEY\"] = \":\" # Set the workspace ID and Lepton API key for authenticating with DGX Cloud Lepton\n", + "os.environ[\"RESOURCE_SHAPE\"] = \"gpu.1xh200\" # Select the appropriate GPU resource for deploying the model\n", + "os.environ[\"NODE_GROUP\"] = \"\" # Select the appropriate node group for deploying the model\n", + "os.environ[\"ACCESS_TOKEN\"] = ACCESS_TOKEN # Set the password for authenticating endpoint requests\n", + "os.environ[\"BASE_MODEL\"] = BASE_MODEL # Set the model to be used for data generation\n", + "os.environ[\"ENDPOINT_NAME\"] = \"nemotron-nano-9b-v2\" # Set the name for the endpoint" + ] + }, + { + "cell_type": "markdown", + "id": "b873dd46", + "metadata": {}, + "source": [ + "Now we authenticate with DGX Cloud Lepton using the provided credentials and deploy the model as an Endpoint." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3bb4eb12-6da8-4ee0-97fd-f35949e37ef5", + "metadata": {}, + "outputs": [], + "source": [ + "%%bash\n", + "\n", + "lep login -c $LEPTON_KEY\n", + "\n", + "lep endpoint create \\\n", + "--resource-shape $RESOURCE_SHAPE \\\n", + "--node-group $NODE_GROUP \\\n", + "--container-image \"vllm/vllm-openai\" \\\n", + "--container-port 8080 \\\n", + "--container-command \"vllm serve ${BASE_MODEL} --port 8080 --trust_remote_code\" \\\n", + "--name $ENDPOINT_NAME \\\n", + "--tokens $ACCESS_TOKEN" + ] + }, + { + "cell_type": "markdown", + "id": "f26998c5", + "metadata": {}, + "source": [ + "Wait for the Endpoint to be available for requests:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f949c89-479a-48b7-831b-84b427174beb", + "metadata": {}, + "outputs": [], + "source": [ + "def wait_for_endpoint(endpoint_name: str, interval: int = 10) -> str:\n", + " command = [\"lep\", \"endpoint\", \"status\", \"-n\", endpoint_name, \"--detail\"]\n", + " while True:\n", + " result = subprocess.run(command, capture_output=True, text=True, check=True)\n", + " for line in result.stdout.split(\"\\n\"):\n", + " if line.startswith(\"State\"):\n", + " _, state = line.strip().rsplit(\" \", maxsplit=1)\n", + " if \"LeptonDeploymentState.Ready\" in state:\n", + " print(\"Endpoint deployed!\")\n", + " else:\n", + " break\n", + " url_match = re.search(r'https://[\\w\\d\\.\\-]+', line)\n", + " if url_match:\n", + " print(f\"URL: {url_match[0]}\")\n", + " return url_match[0]\n", + " print(f\"Waiting for endpoint {endpoint_name} to be ready...\")\n", + " time.sleep(interval)\n", + "\n", + "endpoint_url = wait_for_endpoint(os.environ[\"ENDPOINT_NAME\"])\n", + "base_url = os.path.join(endpoint_url, \"v1\")" + ] + }, + { + "cell_type": "markdown", + "id": "b20bc968", + "metadata": {}, + "source": [ + "### OpenAI Client\n", + "\n", + "Now we're going to:\n", + "1. Initialize OpenAI's client.\n", + "2. Configure the LLM parameters.\n", + "3. Perform a request, and print the LLM response.\n", + "\n", + "
\n", + "NOTE: As stated above, this notebook relies on a DGX Cloud Lepton Endpoint, but you can instead deploy with an existing OpenAI-compatible API service, such as `build.nvidia.com`. If desired, generate a personal API key with:
\n", + "1. Login (or sign up) through build.nvidia.com.
\n", + "2. Click the View Code button and then Generate API Key available on the nvidia/nemotron-nano-9b-v2 model.

\n", + "\n", + "Next, update `base_url` to be `https://integrate.api.nvidia.com/v1` and `ACCESS_TOKEN` to your personal API key that was generated.\n", + "
\n", + "\n", + "" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2c146a96", + "metadata": {}, + "outputs": [], + "source": [ + "from openai import OpenAI\n", + "\n", + "#ACCESS_TOKEN = \"my-nv-api-key\" # Optional: Use NVIDIA's API service instead of the deployed endpoint\n", + "\n", + "# Initialize the OpenAI client for NVIDIA's API\n", + "client = OpenAI(\n", + " #base_url = \"https://integrate.api.nvidia.com/v1\", # Optional: Use NVIDIA's API service instead of the deployed endpoint\n", + " base_url = base_url,\n", + " api_key = ACCESS_TOKEN\n", + ")\n", + "\n", + "# Example: Generate a limerick about GPU computing\n", + "completion = client.chat.completions.create(\n", + " model=BASE_MODEL,\n", + " messages=[{\"role\":\"user\",\"content\":\"Write a limerick about the wonders of GPU computing.\"}],\n", + " temperature=0.6,\n", + " top_p=0.95,\n", + " max_tokens=2048,\n", + " frequency_penalty=0,\n", + " presence_penalty=0,\n", + " stream=True,\n", + " extra_body={\n", + " \"min_thinking_tokens\": 1024,\n", + " \"max_thinking_tokens\": 2048\n", + " }\n", + ")\n", + "\n", + "# Print the streamed response\n", + "for chunk in completion:\n", + " reasoning = getattr(chunk.choices[0].delta, \"reasoning_content\", None)\n", + " if reasoning:\n", + " print(reasoning, end=\"\")\n", + " if chunk.choices[0].delta.content is not None:\n", + " print(chunk.choices[0].delta.content, end=\"\")" + ] + }, + { + "cell_type": "markdown", + "id": "fee40876", + "metadata": {}, + "source": [ + "### Prompt Templates\n", + "\n", + "To generate questions and responses, we need a prompt template so that the model can understand how to generate the data.\n", + "\n", + "We will use:\n", + "* A prompt template to generate subtopics from a user provided topic\n", + "* A prompt template to generate questions for a given subtopic\n", + "* A prompt template to generate responses for a given question" + ] + }, + { + "cell_type": "markdown", + "id": "82172dd4", + "metadata": {}, + "source": [ + "#### Topics and Subtopics" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "32b0a989", + "metadata": {}, + "outputs": [], + "source": [ + "TOPIC_GENERATION_PROMPT_TEMPLATE = \"\"\"\\\n", + "Given a topic, generate a list of {n_subtopics} subtopics that are related to the topic.\n", + "\n", + "The topic is: {topic}\n", + "\n", + "The list must be without numbers, and without any description of the subtopics. The subtopics should be separated by a comma. There must be no other text than the list.\n", + "\"\"\"" + ] + }, + { + "cell_type": "markdown", + "id": "d6b039d3", + "metadata": {}, + "source": [ + "We can use `AsyncOpenAI` to allow nested event loops for async code execution." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "431b7983", + "metadata": {}, + "outputs": [], + "source": [ + "from openai import AsyncOpenAI\n", + "\n", + "client = AsyncOpenAI(\n", + " base_url = base_url,\n", + " api_key = ACCESS_TOKEN\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "99a5b4cb", + "metadata": {}, + "outputs": [], + "source": [ + "async def generate_subtopics(client, topic, n_subtopics): \n", + " prompt = TOPIC_GENERATION_PROMPT_TEMPLATE.format(topic=topic, n_subtopics=n_subtopics)\n", + " return await client.chat.completions.create(\n", + " model=BASE_MODEL,\n", + " messages=[\n", + " {\"role\": \"user\", \"content\": prompt},\n", + " ],\n", + " temperature=0.2,\n", + " top_p=0.7,\n", + " max_tokens=1024,\n", + " )" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4a56630b", + "metadata": {}, + "outputs": [], + "source": [ + "# Example topic and number of subtopics to generate\n", + "# (You can change these values as needed)\n", + "topic = \"Wales\"\n", + "n_subtopics = 5" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8b6bf777", + "metadata": {}, + "outputs": [], + "source": [ + "responses = await generate_subtopics(client, topic=topic, n_subtopics=n_subtopics)\n", + "nonreasoning_answer = re.sub(r'.*', \"\", responses.choices[0].message.content, flags=re.DOTALL).strip()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b99a8823", + "metadata": {}, + "outputs": [], + "source": [ + "print(nonreasoning_answer)" + ] + }, + { + "cell_type": "markdown", + "id": "9fb3545b", + "metadata": {}, + "source": [ + "Now we can store the generated subtopics as a list for the next stage." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e239af15", + "metadata": {}, + "outputs": [], + "source": [ + "subtopic_list = nonreasoning_answer.split(\",\")" + ] + }, + { + "cell_type": "markdown", + "id": "53116ebb", + "metadata": {}, + "source": [ + "#### Generating Questions\n", + "\n", + "With a list of subtopics, the next step is to generate a set of questions for each subtopic. The following code defines a function to batch-generate questions using the LLM." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "541bb24d", + "metadata": {}, + "outputs": [], + "source": [ + "QUESTION_PROMPT_TEMPLATE = \"\"\"\\\n", + "Given a topic, generate {n_questions} questions that could be asked about that topic. Your response should be in a list format.\n", + "\n", + "The topic is: {sub_topic}\n", + "\n", + "The list must be without numbers. The questions should be separated by a newline character. There must be no other text than the list.\n", + "\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "937bc8c8", + "metadata": {}, + "outputs": [], + "source": [ + "async def generate_questions(client, sub_topic, n_questions):\n", + " prompt = QUESTION_PROMPT_TEMPLATE.format(sub_topic=sub_topic, n_questions=n_questions)\n", + " response = await client.chat.completions.create(\n", + " model=BASE_MODEL,\n", + " messages=[\n", + " {\"role\": \"system\", \"content\": \"/no_think\"},\n", + " {\"role\": \"user\", \"content\": prompt},\n", + " ],\n", + " temperature=0.2,\n", + " top_p=0.7,\n", + " max_tokens=1024,\n", + " )\n", + " return response.choices[0].message.content" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2fad8dc0", + "metadata": {}, + "outputs": [], + "source": [ + "import asyncio\n", + "\n", + "async def question_generator(client, subtopic_list, n_question): \n", + " tasks = [generate_questions(client, subtopic, n_question) for subtopic in subtopic_list]\n", + " return await asyncio.gather(*tasks)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4dc2fae0", + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "n_questions = 5\n", + "question_list = await question_generator(client, subtopic_list, n_questions)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "85c6a7a2", + "metadata": {}, + "outputs": [], + "source": [ + "question_list" + ] + }, + { + "cell_type": "markdown", + "id": "d471e352", + "metadata": {}, + "source": [ + "Now we can convert the questions into a single long list for downstream response generation." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8906c80e", + "metadata": {}, + "outputs": [], + "source": [ + "question_list_formatted = []\n", + "\n", + "for question_set in question_list:\n", + " question_list_formatted += question_set.split(\"\\n\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4341f631", + "metadata": {}, + "outputs": [], + "source": [ + "len(question_list_formatted)" + ] + }, + { + "cell_type": "markdown", + "id": "f573900e", + "metadata": {}, + "source": [ + "#### Generating Responses to Questions\n", + "\n", + "Using the question list, we can prompt `Nemotron-nano-9b-v2` to generate multiple responses for each question. \n", + "\n", + "Note: This section includes concurrency control to avoid rate limit errors using the API." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3d44f2d6", + "metadata": {}, + "outputs": [], + "source": [ + "RESPONSE_PROMPT_TEMPLATE = \"\"\"\\\n", + "Given a question, generate 2 responses that could be given to that question. Your response should be in a list format.\n", + "\n", + "The question is: {question}\n", + "\n", + "The list must be in the format:\n", + "\n", + "RESPONSE A: Response A text here\n", + "RESPONSE B: Response B text here\n", + "\"\"\"" + ] + }, + { + "cell_type": "markdown", + "id": "c2f072ce", + "metadata": {}, + "source": [ + "**Tip:** Limit concurrency with a semaphore to avoid API rate limit errors when generating many responses in parallel." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "20f1b99d", + "metadata": {}, + "outputs": [], + "source": [ + "async def generate_response(client, question, sem=None):\n", + " prompt = RESPONSE_PROMPT_TEMPLATE.format(question=question)\n", + " async with sem or asyncio.Semaphore(1): \n", + " response = await client.chat.completions.create(\n", + " model=BASE_MODEL,\n", + " messages=[{\"role\": \"user\", \"content\": prompt}],\n", + " temperature=0.2,\n", + " top_p=0.7,\n", + " max_tokens=1024,\n", + " )\n", + " return response.choices[0].message.content" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "24a325d1", + "metadata": {}, + "outputs": [], + "source": [ + "async def response_generator(client, question_list, max_concurrent=5):\n", + " sem = asyncio.Semaphore(max_concurrent) if max_concurrent else None\n", + " tasks = [generate_response(client, question, sem) for question in question_list]\n", + " return await asyncio.gather(*tasks)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3fc3e91c", + "metadata": {}, + "outputs": [], + "source": [ + "question_response_list = await response_generator(client, question_list_formatted, max_concurrent=5)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "76edbda2", + "metadata": {}, + "outputs": [], + "source": [ + "question_response_pair_list = []\n", + "\n", + "for question, response_set in zip(question_list_formatted, question_response_list, strict=False):\n", + " question_response_pair_list.append(\n", + " {\n", + " \"question\": question,\n", + " \"responses\": {\n", + " \"response_a\": {\"response\": response_set.split(\"RESPONSE B:\")[0].replace(\"RESPONSE A:\", \"\").strip()},\n", + " \"response_b\": {\"response\": response_set.split(\"RESPONSE B:\")[-1].split(\"\\n\\n\")[0].strip()},\n", + " },\n", + " },\n", + " )" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4e834b46", + "metadata": {}, + "outputs": [], + "source": [ + "# Save the generated question-response pairs to a JSONL file for downstream use.\n", + "import json\n", + "\n", + "save_path = os.path.join(SAVE_DIRECTORY, \"synthetic_data.jsonl\")\n", + "\n", + "with open(save_path, \"w\") as f:\n", + " for item in question_response_pair_list:\n", + " f.write(json.dumps(item))\n", + " f.write(\"\\n\")\n", + "\n", + "print(f\"Saved {len(question_response_pair_list)} question-response pairs to {save_path}\")" + ] + }, + { + "cell_type": "markdown", + "id": "03c49613", + "metadata": {}, + "source": [ + "### Math Problems\n", + "\n", + "We can also generate math problems based on a specific subtopic, using a custom parser." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5df37786", + "metadata": {}, + "outputs": [], + "source": [ + "MATH_PROMPT_TEMPLATE = (\n", + " \"Create {n_problems} diverse mathematics problems related to the topic '{topic}' \"\n", + " \"or solvable using concepts from '{topic}'. Provide your response as a numbered list, \"\n", + " \"and include both the problem description and its solution. Format your response as follows: \"\n", + " \"((###1###)) >>>Problem<<<: [Description of the first problem]. >>>Solution<<<: [Solution to the first problem].\\n\"\n", + " \"((###2###)) >>>Problem<<<: [Description of the second problem]. >>>Solution<<<: [Solution to the second problem].\\n\"\n", + " \"Only include the problems and their solutions—no additional text.\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e8f4c408", + "metadata": {}, + "outputs": [], + "source": [ + "n_problems = 5\n", + "topic = \"Algebra\"" + ] + }, + { + "cell_type": "markdown", + "id": "06fa0b23", + "metadata": {}, + "source": [ + " In this case, we will use a customer parser to clean our list of questions/answers." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cc8a83dc", + "metadata": {}, + "outputs": [], + "source": [ + "from typing import List, Tuple\n", + "\n", + "def parse_math_llm_response(\n", + " llm_response: str, tag_replacements: dict\n", + " ) -> List[Tuple[str, str]]:\n", + " \"\"\"\n", + " Expects response from LLM to be wrapped between 2 distinct tags (>>>Problem<<<: and >>>Solution<<<:).\n", + " Uses regex to extract each problem-solution pair, then cleans and replaces original tags with\n", + " more user-friendly labels.\n", + " \"\"\"\n", + " \n", + " first_tag, second_tag = list(tag_replacements.keys())[:2]\n", + "\n", + " pattern = rf\"{re.escape(first_tag)}(.*?){re.escape(second_tag)}(.*?)(?=\\(\\(###|\\Z)\"\n", + " matches = re.findall(pattern, llm_response, re.DOTALL)\n", + "\n", + " problem_solution_pairs = []\n", + " for problem, solution in matches:\n", + " for tag, replacement in tag_replacements.items():\n", + " problem = problem.replace(tag, replacement).strip()\n", + " solution = solution.replace(tag, replacement).strip()\n", + " problem_solution_pairs.append((problem.strip(), solution.strip()))\n", + "\n", + " return problem_solution_pairs\n", + "\n", + "\n", + "async def generate_math(\n", + " client,\n", + " topic: str,\n", + " n_problems: int,\n", + " math_prompt_template: str,\n", + " tag_replacements: dict,\n", + " n_retries: int = 3,\n", + ") -> List[Tuple[str, str]]:\n", + " \"\"\"\n", + " Async function to generate math problems and parse them into (problem, solution) pairs.\n", + " \"\"\"\n", + " math_problems = []\n", + " prompt = math_prompt_template.format(topic=topic, n_problems=n_problems)\n", + "\n", + " for attempt in range(n_retries):\n", + " try:\n", + " response = await client.chat.completions.create(\n", + " model=BASE_MODEL,\n", + " messages=[{\"role\": \"user\", \"content\": prompt}],\n", + " temperature=0.1,\n", + " top_p=0.9,\n", + " max_tokens=1024,\n", + " )\n", + "\n", + " # Extract raw LLM text\n", + " llm_response = response.choices[0].message.content\n", + "\n", + " # Parse with your existing parser\n", + " math_problems = parse_math_llm_response(llm_response, tag_replacements)\n", + "\n", + " break # success, exit retries\n", + "\n", + " except Exception as e:\n", + " print(f\"Attempt {attempt+1}/{n_retries} failed: {e}\")\n", + "\n", + " return math_problems\n", + "\n", + "\n", + "# Example usage\n", + "tag_replacements = {\n", + " \">>>Problem<<<:\": \"Problem:\",\n", + " \">>>Solution<<<:\": \"Solution:\",\n", + "}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "42a7d224", + "metadata": {}, + "outputs": [], + "source": [ + "math_problems = await generate_math(client, topic, n_problems, MATH_PROMPT_TEMPLATE, tag_replacements)\n", + "\n", + "for idx, (problem, solution) in enumerate(math_problems):\n", + " print(f\"Problem {idx+1}: {problem}\")\n", + " print(f\"Solution {idx+1}: {solution}\\n\")" + ] + }, + { + "cell_type": "markdown", + "id": "6367ef60", + "metadata": {}, + "source": [ + "### Summary & Next Steps\n", + "\n", + "- You have now generated a synthetic chat QA dataset using NVIDIA's Nemotron LLMs.\n", + "- The data is saved in `synthetic_data.jsonl` and ready for use in reward modeling or DPO training.\n", + "- Next steps: Evaluate the data, filter for quality, and consider augmenting with real-world data for best results.\n", + "\n", + "---" + ] + } + ], + "metadata": { + "container_image": "nvcr.io/nvidia/nemo:25.07", + "description": "Tutorial on generating synthetic datasets using NIMs for inference", + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.3" + }, + "title": "Data Curation with NIMs: Synthetic Chat QA Generation" + }, + "nbformat": 4, + "nbformat_minor": 5 +}