We will accomplish the following in this tutorial:
- Setup with Docker Compose
- Connect to a datasource
- Create a dataset to monitor
- Generate expectations
- Validate our dataset
1. Clone Swiple GitHub Repository
Clone Swiple GitHub Repository in your terminal with the following command:
git clone https://github.com/Swiple/swiple.git
git clone email@example.com:Swiple/swiple.git
Once the command completes successfully, you should see a new
swiple folder in your current directory, navigate to it.
2. Launch Swiple with Docker Compose
When working on the
main branch, run the following commands:
docker compose -f docker-compose-non-dev.yaml pull
docker compose -f docker-compose-non-dev.yaml up
This will start the following services:
- Swiple API
- Swiple UI
- Swiple Scheduler
- Swiple Setup
- OpenSearch Cluster
- PostgreSQL (Sample Data)
You should start to see a wall of logging output from the containers being launched on your machine. Once the output slows, you can navigate to http://127.0.0.1:3000/login to see the app running.
3. Sign in to Swiple at http://127.0.0.1:3000/login
4. Connect to Sample Database
Included in the running services will be a PostgreSQL database with sample data. You can use this data to give Swiple a test run.
Data Sources in the sidebar followed by the
+ Data Source button.
4.2. Fill in the details of the datasource as shown below then click
You should see your new datasource in the table of data sources! 👏 😎
5. Create a Dataset
Datasets in the sidebar followed by the
+ Dataset button.
5.2. Fill in the details of the data as shown below then click
You should see your new dataset in the table of dataset! 👏 😎
6. Generate and Add Expectations
6.1. Click on "v_orders" to navigate to the dataset view.
6.2. Click on the "Suggestions" tab, then click on "Generate Suggestions".
The dataset you have selected will be analyzed and expectations will be suggested.
6.3. Once a list of suggestions appear, enable the first 4 suggestions.
7. Validate Data
7.1. Navigate to the "Expectations" tab, then click "Run Expectations".
At this point, you should see the list of expectations you enabled, whether they passed or failed validation, the history of all validation runs, and the documentation for each expectation! 👏 😎