feat: add dbt profile init command with type-safe models - #6
Conversation
Add `brix dbt profile init` command to initialize dbt profiles from a bundled template. Features: - Pydantic models for type-safe profile parsing (DbtProfiles, DuckDbOutput) - Template loading via importlib.resources for package-bundled templates - Default DuckDB template for local development - BRIX_DBT_PROFILE_PATH env var to override default ~/.dbt/profiles.yml - --force flag to overwrite existing profiles - `brix dbt profile show` to display current profile configuration Also adds comprehensive unit tests for models, service layer, and CLI.
- Remove separate test.yml workflow (redundant with pre-commit pytest hook) - Add coverage reporting to pytest pre-commit hook - Add codecov upload step to pre-commit workflow
…re-commit workflow - Updated pytest entry in pre-commit config to remove coverage reporting. - Removed the codecov upload step from the GitHub Actions workflow.
- Added a new module for profile editing that supports CRUD operations for dbt profiles and outputs. - Introduced interactive prompts for managing profiles and outputs using the questionary library. - Enhanced the dbt command interface to include an `edit` command for modifying profiles and outputs. - Updated existing profile management commands to integrate with the new editing functionality. - Added comprehensive unit tests for the new profile editor functionality.
- Introduced a new profile management service for handling dbt profiles, including loading templates and validating profiles. - Added a dedicated profile editor module to support CRUD operations for profiles and outputs. - Created a new interactive prompt system for managing profiles and outputs using questionary. - Implemented Pydantic models for type-safe profile configuration. - Updated existing modules to integrate with the new profile management structure. - Added comprehensive unit tests for the new functionality.
|
/review |
PR Review: Add dbt profile init command with type-safe modelsSummaryThis PR adds a comprehensive profile management system for dbt with CLI commands, type-safe Pydantic models, and both interactive and non-interactive editing modes. Overall, this is a well-structured and thoughtful implementation with good separation of concerns and comprehensive test coverage. Code Quality & Best Practices ✅Strengths
Areas for Improvement1. Security Concern: YAML Deserialization (Medium Priority)Location: def from_yaml(cls, content: str) -> DbtProfiles:
import yaml
try:
data = yaml.safe_load(content) # Good - using safe_load✅ Good: Using However, there's a potential issue in the YAML serialization: Location: def to_yaml(self) -> str:
import yaml
return yaml.dump(
self.root,
default_flow_style=False,
allow_unicode=True,
sort_keys=False,
)Issue: The Recommendation: def to_yaml(self) -> str:
import yaml
return yaml.dump(
self.root,
Dumper=yaml.SafeDumper, # Add explicit safe dumper
default_flow_style=False,
allow_unicode=True,
sort_keys=False,
)2. Potential Bug: Missing Validation After Output DeletionLocation: if len(profiles.root[profile_name].outputs) == 1:
msg = f"Cannot delete last output from profile '{profile_name}'. Delete the profile instead."
raise ValueError(msg)Issue: This prevents deleting the last output, but doesn't validate if the current Recommendation: Add validation: def delete_output(profiles: DbtProfiles, profile_name: str, output_name: str) -> DbtProfiles:
# ... existing checks ...
# Check if deleting the current target
profile = profiles.root[profile_name]
if profile.target == output_name:
msg = f"Cannot delete output '{output_name}' as it is the current target. Change target first."
raise ValueError(msg)
if len(profile.outputs) == 1:
msg = f"Cannot delete last output from profile '{profile_name}'. Delete the profile instead."
raise ValueError(msg)3. Code Duplication in CLI Command HandlerLocation: The # Current approach
def _dispatch_cli_action(action, ...):
if action == "add-profile":
_handle_add_profile(...)
elif action == "edit-profile":
_handle_edit_profile(...)
# ... etcRecommendation: ACTION_HANDLERS = {
"add-profile": _handle_add_profile,
"edit-profile": _handle_edit_profile,
"delete-profile": _handle_delete_profile_cli,
"add-output": _handle_add_output_cli,
"edit-output": _handle_edit_output_cli,
"delete-output": _handle_delete_output_cli,
}
def _dispatch_cli_action(action: ActionType, ...):
handler = ACTION_HANDLERS.get(action)
if handler:
handler(profiles, target_path, profile, output, target, path_value, threads, force)This is more maintainable and follows DRY principles. 4. Missing Input ValidationLocation: def update_output(profiles, profile_name, output_name, *, path=None, threads=None):
# ... checks ...
if path is not None:
output.path = path
if threads is not None:
output.threads = threadsIssue: No validation on Recommendation: if threads is not None:
if threads < 1:
msg = "threads must be a positive integer"
raise ValueError(msg)
output.threads = threads5. Inconsistent Error Handling PatternLocation: try:
result = init_profile(profile_path=profile_path, force=force)
typer.echo(result.message)
except ProfileExistsError as e:
typer.echo(str(e), err=True)
raise typer.Exit(1) from None # from None suppresses traceback
except FileNotFoundError as e:
typer.echo(f"Template error: {e}", err=True)
raise typer.Exit(1) from NoneThe Performance Considerations ⚡1. Template Loading EfficiencyLocation: def load_template(name: str = "default.yml") -> str:
import importlib.resources as ir
template_text = ir.files("brix.modules.dbt.profile.templates").joinpath(name).read_text()✅ Good: Using 2. File I/O PatternThe code reads/writes YAML files synchronously, which is appropriate for CLI usage. No concerns here. 3. Pydantic Model PerformanceThe use of Security Concerns 🔒1. Path Traversal Protection ✅Location: def get_default_profile_path() -> Path:
profile_path_str = os.environ.get("BRIX_DBT_PROFILE_PATH")
if profile_path_str:
return Path(profile_path_str).expanduser().resolve()
return Path.home() / ".dbt" / "profiles.yml"✅ Good: Using 2. File PermissionsLocation: target_path.write_text(yaml_content)Minor concern: No explicit file permissions set. dbt profiles can contain database credentials, so consider: target_path.write_text(yaml_content)
target_path.chmod(0o600) # Owner read/write only3. Command InjectionNo shell commands or subprocess calls detected. ✅ Safe. Test Coverage 📊Strengths
Missing Test Cases
RecommendationAdd tests for the interactive prompts module, perhaps using Additional Observations1. Deleted Test Workflow ❓The PR deletes Recommendation: If the workflow was moved elsewhere, please clarify. Otherwise, consider restoring it. 2. questionary DependencyA new dependency
3. Whitespace-Only ChangeLocation: - entry: uv run pytest
+ entry: uv run pytest Trailing whitespace added - likely unintentional. Run 4. pyproject.toml FormattingThe reformatting of Recommendations SummaryMust Fix (Before Merge)
Should Fix (High Priority)
Nice to Have (Medium Priority)
ConclusionThis is a high-quality PR with thoughtful design and good engineering practices. The code is well-structured, follows project conventions, and includes comprehensive tests. The concerns raised are mostly minor improvements and defensive programming suggestions. Recommendation: Approve with minor revisions (address the "Must Fix" items above). Great work! 🎉 Generated with Claude Code |
…ut configuration updates - Added support for Databricks adapter in dbt profile management, including new Pydantic models for DatabricksOutput. - Implemented validation for Databricks authentication methods and connection settings. - Updated existing output management functions to handle both DuckDB and Databricks configurations. - Enhanced interactive prompts for editing profiles and outputs to include Databricks-specific fields. - Added unit tests for new Databricks functionality and validation rules. - Improved logging for error handling in profile management commands.
…ails - Removed redundant information about CI processes and clarified code style guidelines. - Renamed "Project Structure" to "Architecture" and added detailed layer separation. - Introduced key patterns for dbt passthrough, profile models, template system, configuration, and logging. - Improved clarity and organization of the architecture section for better understanding of the project structure.
Add
brix dbt profile initcommand to initialize dbt profiles from abundled template. Features:
brix dbt profile showto display current profile configurationAlso adds comprehensive unit tests for models, service layer, and CLI.