🚀 ZenML 0.80.0: Workspace Hierarchy & Performance Boost
We're thrilled to announce the release of ZenML 0.80.0, one of our most significant updates to date! This release introduces fundamental architectural changes, substantial performance improvements, and enhanced resource management capabilities that benefit both our open-source users and ZenML Pro subscribers in distinct ways.
🌟 Pro Features: Workspace Hierarchy & Projects
- Complete refactoring of tenants into workspaces with project subdivision
- Enhanced RBAC with new default roles for improved permission management
- Supercharged tagging system with multi-tag filtering and exclusive tags
- Completely redesigned dashboard with tab-based navigation and faster load times
🏃 Performance Improvements for All Users
- Significantly faster CLI performance
- Better GitLab repository support with improved URL parsing
- Support for environment variables in KubernetesPodSettings
- Fixed SkyPilot Orchestrator with upgraded dependencies
- Improved Kubernetes orchestrator security with API tokens as secrets
- Enhanced Weights & Biases integration with Weave
- GCP service connector project ID override capability
- Fixed ACR support in Azure service connector
- Persistent resource support for Vertex orchestrator
📚 Brand New Documentation
- Improved navigation with clear site sections
- Dedicated ZenML Pro documentation area
- Enhanced search and AI assistance features
📋 Upgrade Notes
Pro users should explore the new workspace hierarchy and projects feature. Resource organization is now more intuitive with workspace-scoped resources (Stacks, Secrets) and project-scoped resources (pipelines, artifacts, models).
Upgrade with: pip install -U zenml
Full details: https://github.com/zenml-io/zenml/releases
Happy modeling! 🎉
🚀 ZenML 0.73.0: Enhanced Pro Features, Multi-Domain Support & Experiment Tracking
We're thrilled to announce ZenML 0.73.0, bringing powerful improvements for enterprise deployments and experiment tracking capabilities!
🏢 Enterprise-Grade Enhancements
- Secure tenant enrollment for self-hosted ZenML Pro deployments
- Advanced CSRF token protection for enhanced security
- Cross-domain authorization flow for multi-domain installations
- Flexible memory configuration for migration pods in Helm deployments
📊 New Experiment Tracking Features
- Vertex AI experiment tracker integration for seamless cloud operations
- Enhanced experiment comparison tooling for deeper insights
- Interactive visualization capabilities for better analysis
- Check out our quick demo to see it in action!
☁️ Cloud & Infrastructure Updates
- Support for latest Airflow KubernetesPodOperator paths
- Revamped Slack alerter implementation for better notifications
- Improved resource reporting with automatic unit conversion
- Enhanced documentation for Kubeflow Pipelines and LLM deployments
🛠️ Quality-of-Life Improvements
- Non-ASCII character support in JSON dumps via environment variables
- Removed gluon from MLflow log suppression list
- Various dashboard fixes for smoother user experience
📋 Upgrade Notes
Before upgrading to 0.73.0:
- Review the on-premises Pro deployment documentation
- Backup your existing configurations
- Test the new features in a non-production environment first
As always, we recommend checking our full changelog for detailed information about all changes and improvements. Join our community channels to share your feedback and experiences with these new features!
Happy modeling! 🎉
Full release notes: https://github.com/zenml-io/zenml/releases/tag/0.73.0
🔍 NEW: Experiment Comparison Tool now available in the dashboard
ZenML now offers powerful experiment tracking capabilities directly in your dashboard (Pro users only). Compare and analyze your pipeline runs with our new Experiment Comparison Tool:
📊 Interactive Table View: Compare metadata and metrics across pipeline runs with automatic change tracking and sorting capabilities
📈 Parallel Coordinates Plot: Visualize relationships between different metadata parameters across multiple runs
🔄 Compare up to 20 pipeline runs simultaneously and analyze any numerical metadata your pipelines generate
🔗 Shareable Views: Configurations are preserved in URLs for easy sharing with team members
This feature is currently in Alpha Preview. Join our Slack to share feedback!
ZenML 0.70.0: Enhanced Artifacts Versioning, Scalability and Metadata Management
The ZenML 0.70.0 release includes a significant number of database schema changes and migrations, which means upgrading to this version will require extra caution. As always, please make sure to make a copy of your production database before upgrading.
Key Changes
Artifact Versioning Improvements: The handling of artifact versions has been improved, including the API improvements like the ability to batch artifact version requests to improve the execution times and more types for the step input/output artifacts, including multiple versions of the same artifact (e.g. model checkpoints), to improve the UX using ZenML UI or while working directly with the API.
Scalability Enhancements: Various scalability improvements have been made, such as reducing unnecessary server requests and incrementing artifact versions server-side. These enhancements are expected to provide significant speed and scale improvements for ZenML users.
Metadata management: Now, all the metadata-creating functions are gathered under one method called
log_metadata. It is possible to call this method with different inputs to log run metadata for artifact versions, model versions, steps and runs.The oneof filtering: This allows to filter entities using a new operator called
oneof. You can use this with IDs (UUID type) or tags (or other string-typed attributes) like thisPipelineRunFilter(tag='oneof:["cats", "dogs"]').Documentation Improvements: The ZenML documentation has been restructured and expanded, including the addition of new sections on finetuning and LLM/ML engineering resources.
Bug Fixes: This release includes several bug fixes, including issues with in-process main module source loading, and more.
Caution: Make sure to back up your data before upgrading!
While this release brings many valuable improvements, the database schema changes and migrations pose a potential risk to users. It is strongly recommended that users:
Test the upgrade on a non-production environment: Before upgrading a production system, test the upgrade process in a non-production environment to identify and address any issues.
Back up your data: Ensure that you have a reliable backup of your ZenML data before attempting the upgrade.
✨ ZenML 0.68.0: Enhanced Dashboard, Client-Side Caching, and Streamlined Experience!
🚀 We're excited to announce the release of ZenML 0.68.0 (and 0.68.1) with powerful improvements to boost your ML workflow productivity:
- 📊 Stack Components Dashboard: Visualize and manage your stack components directly from the dashboard interface
- 🧠 Filters: ZenML Pro users are now able to filter and sort all resources in the dashboard!
- ⚡ Client-Side Caching: Dramatically improved performance with client-side computation for cached steps, reducing remote orchestrator spin-up time and costs
- 🔄 Unified Onboarding: Combined starter and production setup into a single, intuitive flow for smoother getting started experience
- 📦 New Artifact Management: Introducing register_artifact function to directly link existing data in the artifact store - perfect for frameworks like PyTorch-Lightning
- 🚢 Updated BentoML Integration: Now supporting version 1.3.5 with improved containerization
Important Changes:
- Python 3.8 support has been discontinued
- Several legacy features including the legacy pipeline and step interface have been removed
- Various CLI commands including zenml stack up/down are now deprecated
📚 Expanded Documentation:
- New guides for Kubernetes per-pod configuration
- Best practices for common stacks
- Azure 1-click dashboard deployment
- Comprehensive ZenML Pro documentation
- Detailed guides for custom Dataset classes and Materializers
Upgrade to ZenML 0.68.x today to access all these improvements! Check out the full release notes on GitHub for more details.
Happy modeling! 🎉
😃 ZenML 0.67.0: Supercharged SageMaker, New DAG Visualizer, and More!
🚀 We're excited to announce the release of ZenML 0.67.0, packed with powerful new features and improvements:
- Enhanced SageMaker Orchestration: Support for TrainingJobs with warm pools, boosting performance and potentially reducing costs.
- Revamped DAG Visualizer: Preview DAGs before pipeline completion and enjoy improved visual clarity.
- Flexible Environment Configuration: Use ${ENV_VARIABLE_NAME} syntax in code and config files for adaptable deployments.
- Improved Cloud Integration: Direct pipeline and log URLs for AWS, Azure, and GCP.
- Kubernetes Support for Skypilot: Run Skypilot orchestrator on Kubernetes clusters.
- Updated Deepchecks Integration: Incorporating the latest Deepchecks features for robust data validation.
- Expanded Documentation: Covering Lightning AI, Kubeflow, Comet, Neptune, Hugging Face deployer, and more.
Upgrade to ZenML 0.67.0 today to supercharge your ML workflows! Check out the full release notes on GitHub for more details.
Happy modeling with ZenML! 🎉
🔥Python 3.12 and slimmer ZenML in the latest ZenML 0.66.0
This release adds support for Python 3.12, which means you can now develop your ZenML pipelines with the latest Python features. Before this release, settings for stack components had to be specified with both the component type as well as the flavor. We simplified this and it is now possible to specify settings just using the component type:
Before:
@pipeline(settings={"orchestrator.sagemaker": SagemakerOrchestratorSettings(...)})
def my_pipeline():
Now:
@pipeline(settings={"orchestrator": SagemakerOrchestratorSettings(...)})
def my_pipeline():
Finally, in order to slim down the ZenML library, we removed the numpy and pandas libraries as dependencies of ZenML. If yourcode uses these libraries, you have to make sure they're installed in your local environment as well as the Docker images that get built to run your pipelines (Use DockerSettings.requirements or DockerSettings.required_integrations).
If you’re a pro user update your tenant today with a single click, and locally run pip install zenml —upgrade to immediately get access to all this and above!
Full release notes here: https://github.com/zenml-io/zenml/releases/0.66.0
🚀 ZenML 0.65.0: Streamlined ML Workflows and Cloud-Scale Operations
Exciting Updates in ZenML's 0.65.0 Release!
We're thrilled to announce the 0.65.0 release of ZenML, packed with new features and improvements to streamline your MLOps workflows:
New Quickstart Experience: Seamlessly transition from local to cloud-scale ML operations with our enhanced onboarding process.
Run Single Step as ZenML Pipeline: Execute individual steps on your active stack, improving flexibility and debugging capabilities.
AzureML Integration Upgrade: Our AzureML Step Operator now supports SDKv2 and leverages Service Connectors for improved cloud integration.
Enhanced Logging: Benefit from improved traceability with new timestamp features in log messages.
Flexible Model Versioning: Utilize templated names for Model Versions, supporting dynamic
{date}and{time}placeholders.Accelerate Decorator: Use the
run_with_acceleratestep wrapper as a Python Decorator for ZenML steps, simplifying integration with accelerate library.
These updates focus on improving developer experience, expanding cloud capabilities, and enhancing flexibility in ML workflows. Update now to take advantage of these powerful new features!
For a complete list of changes and improvements, check out our full release notes on GitHub.
Happy modeling with ZenML!
👌 ZenML 0.64.0 Release: ZenML in Notebook and No More Persistent Docker Builds
Exciting updates in our latest release:
Remote Notebook Integration: Run notebook-defined steps with remote orchestrators.
Optimized Docker Builds: Faster development with code uploads to artifact store.
AzureML Orchestrator Support: Expanded cloud platform integration.
Terraform Modules: Easily provision MLOps stacks across major cloud providers.
These features bridge the gap between experimentation and production, accelerate development cycles, and simplify cloud infrastructure setup.
Update now to experience streamlined MLOps workflows!
For details, visit the recent blog post and release notes.
🫶 v0.63.0 is out! Azure support for stack registraton + no more pipeline versions
Moving forward from the last two releases, we have further improved the 1-click deployment tool and the stack wizard by adding support for Azure.
Moreover, we implemented a new step operator that allows you to run individual steps of your pipeline in Kubernetes pods.
Lastly, we have simplified our pipeline models by removing their versions. No migration is required but if you were using the API, please take out references to pipeline versions, and reference runs directly from now on.
See full release notes here: https://github.com/zenml-io/zenml/releases/tag/0.63.0
