Meng Lim (@menglim)

Learning to Fall: The Beginning of My Judo Journey

2025-09-11T10:43:15.591582Z

Learning to Fall: The Beginning of My Judo Journey

I still remember my first day on the tatami. I walked into the dojo, not knowing what to expect. The room was full of young children—toddlers, really—laughing and gleefully throwing each other around. Watching them tumble and roll with such carefree joy was both hilarious and oddly mesmerizing.

Among the other first-timers, I met our sensei for the first time. He kindly lent us Judo Gis and patiently taught us how to wear them properly: left side over right. In Japanese tradition, crossing the right side over the left is reserved specifically for the deceased. He then showed us how to tie our belts, and without much further instruction, we followed along with running—forwards, backwards, and sideways—rolling our wrists and neck, and stretching our chests.

Being no stranger to the gym, I initially thought the warm-up felt manageable—until we reached the next stage. Forward rolls, backward rolls, handstand rolls, backward handstand rolls, cartwheels, and a host of other movements that, while not entirely foreign, I had never attempted. Even a year later, the 15-minute warm-up still humbles me. No matter how many times I repeat it, the spinning, rolling, and tumbling always leaves me a dizzy.

The first real lesson was ukemi, or break-falling—the art of falling safely. For the next 45 minutes, we practiced all types of ukemi, slapping the mat repeatedly, unsure at first how it would actually reduce the impact. By the end, though, I could feel my body slowly learning to absorb the falls.

Eventually, we progressed to our first throw: O-Goshi (Major Hip Throw). The sensei demonstrated it, and as the Uke (the partner receiving the throw) hit the mat with a satisfying thud, awe spread across everyone’s faces. Quiet murmurs of “ooooh” and “wow” rippled through the group. The throw looked smooth, effortless—and yet utterly devastating.

Breaking down the technique revealed its true complexity. There are four main principles in any Judo throw, simplified here:

Kumi-kata (Establishing Grips): Securing a dominant grip on the opponent’s gi to control posture and movement.
Kuzushi (Breaking Balance): Using the grips and body movement to off-balance the opponent, making them vulnerable.
Tsukuri (Entry/Positioning): Positioning the body correctly for the throw while the opponent remains off-balance.
Kake (Execution): Completing the throw using leverage and momentum to bring the opponent to the mat.

If you are keen to see the breakdown for Ogoshi - watch this Youtube tutorial Watch O-goshi tutorial or watch it from the official Kodokan archives Watch Kodokan Ogoshi

If you like to see it competition environment aka in the Olympics - Watch here

At the end of class, one of the senior judoka threw us onto a thick padded mattress to practice break-falls before experiencing a full throw on the mat. Nervous anticipation mixed with exhilaration—it was the first real taste of Judo’s blend of discipline, respect, and controlled chaos.

Looking back, that first day taught me more than technique. It taught me humility, patience, and the importance of showing up—even when the mats feel intimidating, the drills exhausting, or the throws impossible. Judo, I realized, isn’t just about throwing—it’s about learning to fall, getting back up, and embracing every moment in between.

Takeaway: Embrace the falls, focus on learning, and celebrate every small step forward.

]]>

Designing a Modern Data Platform: Integrating AWS AppFlow, Fivetran, Snowflake, dbt, and Alation

2025-09-11T02:06:40.286724Z

Designing a Modern Data Platform: Integrating AWS AppFlow, Fivetran, Snowflake, dbt, and Alation

Why Modern Data Platforms Matter

Data isn’t just an asset — it powers smarter decisions, drives operational efficiency, enhances customer experiences, and enables strategic growth. Companies generate vast amounts of information daily, from customer records and marketing campaigns to app usage logs. But raw data alone isn’t enough; turning it into insights that are trustworthy, actionable, and strategically valuable is what drives real business impact. This is where a modern data platform comes in.

Modern platforms unify fragmented data, enabling organizations to extract value efficiently, govern information responsibly, and operationalize insights across teams. They go beyond centralized storage to create an ecosystem where analytics can drive real-time, data-informed decisions.

The Big Picture

A modern data platform is more than a collection of tools — it’s a framework built around five pillars:

Data Ingestion – Fivetran, AWS AppFlow, Amazon DMS: Centralize raw data from SaaS apps, Postgres databases, APIs, and streams.
Storage & Modeling – Snowflake: Structure and organize data for performance, clarity, and trust.
Transformation & Quality – dbt: Transform raw data into reliable, analytics-ready datasets while enforcing quality.
Governance & Discovery – Alation: Build trust, ensure lineage, and support self-service discovery.
Operational Activation – AWS AppFlow (Reverse ETL / Census ETL): Push insights back into business systems, including AI/ML-driven predictions and recommendations, enabling both real-time and predictive decision-making.

Figure-1 Modern Data Platform Architecture

Where Insights Meet Action

The real value of a modern platform emerges when all components interact seamlessly. Data flows from Snowflake into dbt for refinement, is cataloged and governed through Alation, and can ultimately be operationalized back into business systems.

Insights aren’t just visualized for human decision-making — they can trigger automated, real-time actions across the organization. Dashboards remain valuable for monitoring trends, KPIs, and business metrics, but the bigger vision is to move insights closer to action, turning data into tangible impact.

Scenario: Sales Opportunity Prioritization

Traditional dashboard approach:

Sales managers review a Tableau or Salesforce dashboard showing opportunity scores, deal stages, and lead engagement.
They manually decide which leads or accounts to prioritize.

Bigger vision / operationalized approach:

A predictive scoring model evaluates every lead daily for likelihood to close.
Scores are automatically pushed into Salesforce via Reverse ETL.
High-priority leads trigger automated workflows:
- Route leads to the appropriate reps.
- Send automated emails or follow-ups.
- Schedule reminders in Salesforce for timely engagement.

Sales dashboards still exist for monitoring pipeline health, but the platform moves the most promising leads into action, reducing response time and improving close rates.
This illustrates how a modern data platform transforms static insights into automated, actionable workflows.

The Heart of the Platform

At the core is Snowflake, which stores raw ingested data following the layered design shown in Figure 1:

Bronze Layer (Raw / Landing): Raw ingested data preserved for traceability.
Silver Layer (Curated / Staged): Cleansed, harmonized, and validated data forming a reliable foundation.
Gold Layer (Mart / Analytics-ready): Fully refined datasets for insights, dashboards, operational use, and ready for data marts or secure sharing with internal/external teams (e.g., private shares or Snowflake Marketplace).

While Snowflake provides the backbone, dbt is where the real magic happens. It transforms, tests, and models raw data into trusted, actionable intelligence. Both dbt and Schemachange are version-controlled in GitHub for collaboration, reproducibility, and auditability. dbt handles transformations, testing, and modeling, while Schemachange generates traceable DDL/DML scripts. Deployment automation is managed separately via GitHub Actions, keeping the platform reliable and consistent.

Orchestration, Observability, and Security

Orchestration: Workflows managed through dbt Cloud Jobs to handle scheduling, dependencies, and retries.
Observability: Logs from dbt and AWS services flow into Splunk and AWS CloudWatch, with Slack alerts for anomalies.
Security & Compliance: RBAC, dynamic masking, IAM roles, CloudFormation provisioning, and Docker ensure secure, auditable, and consistent environments.

Fivetran and AWS AppFlow: Data Ingestion

Fivetran and AWS AppFlow ensure continuous, reliable data flow into the platform. Fivetran provides managed connectors for apps like Salesforce and Marketo, while AppFlow supports flexible integrations, reverse ETL, and data from relational sources like Postgres.

Alation: Governance and Discovery

Alation enables trust and compliance with:

Automated dataset cataloging
Lineage tracking across ingestion and transformations
Stewardship workflows for critical datasets

Employees can explore datasets confidently, understanding the meaning and context of each column, supporting self-service analytics.

Operational Activation: Reverse ETL

Curated insights become actionable when fed into operational systems. AWS AppFlow moves them into CRMs, marketing platforms, or support tools. For example, customer lifetime value metrics synced into Salesforce enable real-time, data-driven decisions.

Delivering Value Across the Organization

Seamless flow: Systems connected end-to-end, minimizing friction.
Trust and governance: Quality checks, lineage tracking, and RBAC ensure reliability.
Scalability: New sources, transformations, and destinations can be added without disruption.
Operational impact: Verified data reaches the right teams, enabling informed, timely decisions.

Trade-offs and Lessons Learned

Performance vs. Cost in Snowflake:
Initially, all warehouses defaulted to M size, and data retention was set to maximum days, which resulted in high costs. Most workloads could have run on XS or S warehouses, and data retention could be tuned per use case.
Lesson: Balancing performance and cost requires deliberate warehouse sizing, retention policies, and monitoring usage patterns.
Fivetran Ingestion vs. Cost and Source Load:
Frequent ingestion from the main Postgres database caused load spikes and increased cost due to MARR and high volumes. Switching to an in-house ingestion strategy using AWS DMS or AppFlow reduced costs and lessened strain on source systems.
Lesson: Evaluate ingestion frequency, revisit ingestion patterns, and impact on source systems to optimize cost and reliability.
Query Performance vs. Model Complexity in dbt:
dbt transformations prioritized clarity and modularity, which sometimes slowed queries on large datasets. Using materialized views improved performance but increased maintenance overhead.
Lesson: Balance maintainable, modular data models with runtime performance, and profile models for production workloads.
RBAC Ownership vs. Cross-team Integration:
Snowflake integrates with OKTA, managed by a separate security team. Initially, duplicating groups and roles for quick access became difficult to manage.
Lesson: Define clear ownership boundaries, specifying which teams manage which roles and groups, reducing duplication and complexity.
Code Visibility vs. Ad-hoc Workarounds:
Team members unfamiliar with Git workflows sometimes manually ran queries to unblock others, which caused the RBAC source of truth to go out of alignment.
Lesson: Emphasize code visibility, version control, and peer review, ensuring all changes are tracked and auditable.

In summary, a modern data platform is more than technology — it’s an enabler of trust, efficiency, and operational impact. By combining ingestion, transformation, governance, observability, and secure operations, organizations can unlock the full potential of their data and turn it into a strategic advantage.

]]>

From Zero to Deployed: A guide to Vite, React, and Github Pages

2025-09-06T02:16:31.773808Z

This guide will walk you through how to Deploy a React app using Vite and Github Pages, which you can serve on your githubUsername.github.io domain for Free 😉, or on a custom domain you own.

My use case: A personal website, but this works for any single-page application.

Note: I initially built this website with a static HTML and CSS, but after the code grew to over 1,000 lines (yikes! I know), I decided to relive my days as a software engineer and write it in React. This guide highlights some of the key challenges I faced deploying React and Vite on Github Pages.

Step 1: Naming Your Repository

As trivial as it is, this first step is more important than it sounds. The name of your GitHub repository directly determines your final URL. To have your website served from https://githubUsername.github.io without a sub-folder, you must name your repository exactly githubUsername.github.io.

For example, if I want the final URL to be https://mengopudding.github.io and my username is mengopudding, I must name my repository mengopudding.github.io. Had I named the repo portfolio, the final URL would be https://mengopudding.github.io/portfolio

Step 2: Create a React App with Vite

While it has been many moons since I’ve written any JavaScript. While I still remember the gist of things (so I thought), the only one command I knew create-react-app, is now considered outdated and there is a new kid on the block: Vite

To get started, open your terminal and run the following command. Choose “React” and choose your flavor when prompted. ~ I went with TypeScript.

npm create vite@latest my-app — —template react-ts

To verify things are working, navigate into your new project and run the standard dev commands:

cd my-app
npm install
npm run dev

You should also see a vite.config.ts file

Step 3: Configure Tailwind CSS

I never used Tailwind in production environment before, the last design system I used was Braid by Seek — but Tailwind seems to be the go-to in the recent years.

You can also follow the official Get started with Vite & Tailwind documentation.

First, install the necessary dependencies for Tailwind CSS.

npm install tailwindcss @tailwindcss/vite

Next, configure the Vite plugin in your vite. See Official docs for configuring your vite plugin.

Your vite.config.ts should look like this.

//vite.config.ts

import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import tailwindcss from '@tailwindcss/vite' // <- added this import

export default defineConfig({
  plugins: [react(),tailwindcss()], // <- added tailwindcss() to the array
})

Next, add the Tailwind directives to your main CSS file. This is often src/index.css or src/App.css.

 /* App.css or index.css */
@import "tailwindcss";

Tailwind uses a Just-in-Time (JIT) compiler to scan your code and generate only the CSS you need, resulting in much smaller, more efficient CSS files.

To verify Tailwind is working, try adding a class like text-red-500

Vite + React

To verify, run npm run dev, visit the localhost and you should see the changes.

Step 4: Configure Vite for Github Pages

This is a critical step that you must get right. By default, Github Pages serves your site from a sub-path (//), but a standard Vite app assumes it’s served from the root (/). You need to tell Vite to build your project with the correct base URL.

Edit your vite.config.ts file and add the base property. See docs for more info.

// vite.config.ts

// If my repo was named https://github.com/mengopudding/mengopudding.github.io
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import tailwindcss from '@tailwindcss/vite'

export default defineConfig({
  base: '/', // <- added line
  plugins: [react(), tailwindcss()],
})

// If my repo was named https://github.com/mengopudding/my-app
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'

export default defineConfig({
  base: '/my-app/', // <- remember to include both slashes wrapped in strings
  plugins: [react(), tailwindcss()],
})

The key here is the base property. If your repository is named my-app, your base path would be '/my-app/'.

For a username.github.io repository, the base path is simply /.

Step 5: Add a Deploy Script

We’ll use a helpful package called gh-pages to automate the build and deployment. First, install the package as a development dependency:

npm install -D gh-pages

Next, add a homepage property and the predeploy and deploy scripts to your package.json. Make sure to replace the URL with your own.

"homepage": "https://mengopudding.github.io", // <- lets Github Pages know what the main landing page is
"scripts": {
  "dev": "vite",
  "build": "vite build",
  "preview": "vite preview",
  "predeploy": "npm run build",
  "deploy": "gh-pages -d dist"
}

The npm run build command takes all of your code and transforms it into a highly efficient bundle,called dist.
The gh-pages -d dist command essentially says, “use the gh-pages tool to deploy the contents of the dist directory to a new branch called gh-pages in my repository.”

Step 6: Build and Deploy

Ensure all your code is committed to your main branch or that your feature branch is merged into main.
Bundle up dist folder with the necessary files.

npm run build

Note: You will need to fix any issues like linting or TS complaints. The build must succeed!

Now, you can run the deploy script. This will build your app and publish the dist directory to the gh-pages branch on GitHub

npm run deploy

Note: This will automatically create a new branch called gh-pages in your repo, it will publish only the dist folder into the gh-pages branch. Verify this by going to your repo.

Step 7: Finalize Github Pages Configuration

Go to your Github repo

Navigate to Settings > Pages > Build and deployment > Source: Deploy from a branch > select gh-pages branch / (root) > Save

— See Github official docs for publishing source for your Github Pages site

Finally! The app will start to deploy, you can track its deployment through the Actions tab. Once deployed it will provide you with a URL to your app and… VOILA! CONGRATULATIONS🎉

Troubleshooting

All these issues can cause your app to not render once it has been deployed.

Check your build: The most common issue is a failed build. The npm run deploy command will fail if your npm run build command fails. Make sure you don’t have any TypeScript errors or other issues.

Incorrect vite.config.js: Have you correctly set the base property?

Missing scripts in package.json:

Have you set the homepage key to your repo?
Have you added the predeploy and deploy script?

If you see errors like MIME type or 404

Wait a few minutes: It can take a few minutes for Github Pages to process and deploy the new changes to their servers.
Check paths to assets: Assets are kept in the public/assets folder, to reference these assets it’s best to use relative path, eg./assets/media/hello_world.png
Cache issue: Sometimes, the browser caches the old version of your site. Clear your site cache and then perform a hard refresh.

Right Click > Inspect > Application > Clear site data > Hard refresh

]]>