I worked on AI-powered search for Internet Directory. It’s working well.

I worked on AI-powered search for Internet Directory. It’s working well.

Have you heard of the flesh eating bacteria in the ocean like at Outer Banks, Florida? Will that be permanent now due to global warming? That is a real bummer! We can no longer get in the ocean? What can we do to destroy that bacteria in the ocean? It’s sad to not go to the ocean any more.
I saw the prime minister of India called Putin his friend, and Putin said he will end the war in exchange for part of Ukraine. That makes me so mad about that.
I worked on adding stages to Jenkins today. They’re all passing. If you can’t figure out a Jenkins problem, let me know. I could share my Jenkinsfile publicly.
By the way, I watched “Zero Day” on Netflix recently, and it is a great show. I highly recommend it. Lots of relevant tech and political stuff!

A real-world case study in why continuous integration is essential for catching environment-specific issues
We’ve all been there. Your code runs perfectly in your local development environment, all tests pass, and everything seems ready for production. But then your CI pipeline fails with a cryptic error that makes no sense given your local success.
This exact scenario happened to us recently while working on the Internet Directory project, and it perfectly illustrates why robust CI/CD pipelines are invaluable for catching issues that local development simply cannot detect.
Our Jenkins CI pipeline started failing with this error:
ModuleNotFoundError: No module named 'app.models.enums'
The confusing part? Everything worked perfectly locally:
app/models/enums.py clearly existedWhen faced with a “works locally, fails in CI” situation, the first instinct is often to blame the CI environment. But Jenkins was actually doing us a huge favor by exposing a critical configuration issue.
Here’s what we discovered through systematic debugging:
# Test the exact import that was failing in Jenkins
python -c "from app.models.enums import SiteSource"
# ✅ Success locally
# Check if file exists
ls app/models/enums.py
# ✅ File exists
# Check git status
git status
# ✅ Working tree clean
Everything looked normal, which made the Jenkins failure even more puzzling.
The breakthrough came when we checked what files were actually tracked by git:
git ls-files | grep enums
# ❌ No output - file not tracked!
Despite the file existing locally and git status showing a clean working tree, the critical enums.py file was never committed to the repository.
The culprit was hiding in our .gitignore file:
# Model files and checkpoints
models/
This innocent-looking line was designed to ignore machine learning model files, but it had an unintended consequence: it was also ignoring our SQLAlchemy model directory at backend/app/models/.
This is a perfect example of why CI environments are so valuable:
Jenkins was essentially performing a clean room test that revealed our git configuration was broken.
This incident highlighted several critical issues that could have caused problems in production:
# Fix the .gitignore to be more specific
- models/
+ /models/
+ backend/ml_models/
# Add the missing files
git add backend/app/models/enums.py
git add backend/app/models/pending_domain.py
git commit -m "Fix missing model files"
To prevent this in the future, we created a simple test script that simulates the Jenkins environment:
#!/bin/bash
# Clear Python cache to simulate fresh environment
find . -name "__pycache__" -type d -exec rm -rf {} +
# Test imports in a fresh Python session
python -c "
import sys
sys.path.insert(0, '.')
from app.models.enums import SiteSource
print('✅ Import successful')
"
This allows developers to catch git configuration issues before they reach CI.
This incident perfectly demonstrates why continuous integration is not just about running tests—it’s about validating that your entire development workflow is sound. Jenkins didn’t just catch a bug; it caught a process failure that could have caused significant problems down the line.
The next time your CI fails with a mysterious error while everything works locally, don’t dismiss it as a “CI problem.” Instead, treat it as valuable feedback about potential issues in your development process, git configuration, or environment assumptions.
Your CI pipeline is often the first line of defense against the dreaded “it works on my machine” syndrome. Trust it, investigate thoroughly, and you’ll often discover issues that would have been much more expensive to fix in production.
Have you experienced similar “works locally, fails in CI” situations? What did they teach you about your development process? Share your stories in the comments below.
I took a backup a few hours ago, and successfully restored a 4.8GB sql file just now after AI dropped a needed table in a migration. With Alembic/FastAPI it autogenerates migrations, and it autogenerated dropping a table and ran it before I could look at it. I have since updated my code so autogeneration of migrations is working properly.
Update: I added an Alembic autogenerate check step on Jenkins so this shouldn’t happen in the future.
I have over 100K sites in the internet directory database and a pull request with 50+ comments from CodeRabbit. Interested in helping? Email me: andy@greenrobot.com



I’m working on an internet directory project. A project to organize the internet. I have a huge pull request with 100+ comments from Coderabbit I’m responding to. Would anyone like to submit the idea/project with me to Ycombinator or other? Looking for a partner! Do you remember or know about Dmoz or Yahoo Internet Directory? Both not available anymore. My stack is FastAPI and React.

Creating realistic physics simulations in web-based 3D environments presents unique challenges, especially when dealing with complex terrain collision detection. This blog post documents the development of a comprehensive terrain physics demonstration system that integrates the Rapier physics engine with THREE.Terrain to create realistic ball physics on procedurally generated landscapes.
Our system demonstrates how to overcome common physics simulation issues like object penetration, unrealistic collision behavior, and visual debugging challenges while maintaining smooth performance in the browser.
The demonstration leverages several key technologies working in harmony:
// Core system initialization
const physics = await RapierPhysics();
const terrainScene = Terrain(terrainOptions);
const heightData = extractHeightDataFromTerrain(terrainScene);
physics.addHeightfield(terrainMesh, segments, segments, heightData, scale);
The architecture follows a clear separation of concerns:
The heart of our collision system uses trimesh colliders, which provide pixel-perfect collision detection against complex terrain geometry:
function addHeightfield(mesh, width, depth, heights, scale) {
// Extract vertices and transform to world coordinates
const geometry = mesh.geometry;
const positions = geometry.attributes.position.array;
const vertices = new Float32Array(positions.length);
// Transform each vertex to world coordinates
mesh.updateMatrixWorld(true);
const worldMatrix = mesh.matrixWorld;
for (let i = 0; i < positions.length; i += 3) {
tempVector.set(positions[i], positions[i + 1], positions[i + 2]);
tempVector.applyMatrix4(worldMatrix);
vertices[i] = tempVector.x;
vertices[i + 1] = tempVector.y;
vertices[i + 2] = tempVector.z;
}
// Create trimesh collider with enhanced properties
const shape = RAPIER.ColliderDesc.trimesh(vertices, indices);
shape.setFriction(0.8);
shape.setRestitution(0.0);
const body = world.createRigidBody(RAPIER.RigidBodyDesc.fixed());
world.createCollider(shape, body);
}
Perfect alignment between visual terrain and physics collision requires extracting height data directly from the THREE.Terrain geometry:
function extractHeightDataFromTerrain() {
const terrainMesh = terrainScene.children[0];
const positions = terrainMesh.geometry.attributes.position.array;
const heightData = new Float32Array(width * depth);
// THREE.Terrain stores height in Z component before rotation
for (let z = 0; z < depth; z++) {
for (let x = 0; x < width; x++) {
const vertexIndex = (z * width + x) * 3;
const height = positions[vertexIndex + 2]; // Z component contains height
heightData[z * width + x] = height;
}
}
return heightData;
}
The wireframe grid overlay provides crucial visual feedback by using the exact same geometry as the terrain:
function createPhysicsDebugVisualization() {
// Clone the exact terrain geometry for perfect alignment
const terrainMesh = terrainScene.children[0];
const debugGeometry = terrainMesh.geometry.clone();
const debugMaterial = new THREE.MeshBasicMaterial({
color: 0x00ff00,
wireframe: true,
transparent: true,
opacity: 0.6,
side: THREE.DoubleSide
});
const debugMesh = new THREE.Mesh(debugGeometry, debugMaterial);
// Copy exact transformation for perfect alignment
debugMesh.position.copy(terrainMesh.position);
debugMesh.rotation.copy(terrainMesh.rotation);
debugMesh.scale.copy(terrainMesh.scale);
debugMesh.position.y += 1.0; // Slight offset to avoid z-fighting
terrainScene.add(debugMesh);
}
This approach ensures the debug visualization perfectly matches the physics collision surface, eliminating any discrepancies between what users see and what the physics engine calculates.
Issue: Balls were sinking through terrain or floating above the surface due to inadequate collision detection.
Root Causes:
Solutions Implemented:
// Increased physics timestep resolution
const physicsTime = INV_MAX_FPS / 4; // 240 FPS instead of 120 FPS
// Enhanced world configuration
world.integrationParameters.maxCcdSubsteps = 8;
world.integrationParameters.erp = 0.8;
// Improved collision properties
const physicsBody = physics.addMesh(ball, mass, restitution, {
friction: 0.8, // Increased from 0.7
linearDamping: 0.001, // Reduced from 0.02
angularDamping: 0.05 // Reduced from 0.1
});
Issue: Balls exhibited “janky” movement with excessive bouncing and unrealistic physics.
Technical Solutions:
const gravity = new Vector3(0.0, -19.62, 0.0); // 2x Earth gravity
linearDamping: 0.001 // 50x reduction in air resistance
if (physicsBody) {
const initialVelocity = { x: 0, y: -10, z: 0 };
physicsBody.setLinvel(initialVelocity, true);
}
const y = 300 + Math.random() * 150; // Higher starting position
Issue: Visual terrain and physics collision surface were misaligned, causing apparent penetration.
Solution: Direct geometry cloning ensures perfect alignment:
// Use exact terrain geometry for physics collision
const physicsTerrainMesh = terrainMesh.clone();
physicsTerrainMesh.position.copy(terrainMesh.position);
physicsTerrainMesh.rotation.copy(terrainMesh.rotation);
physicsTerrainMesh.scale.copy(terrainMesh.scale);
physics.addHeightfield(physicsTerrainMesh, segments, segments, heightData, scale);
We eliminated the physics debug toggle button and made the grid overlay always visible by default:
// Grid is always created and visible when physics initializes
createPhysicsDebugVisualization();
debugMesh.visible = true; // Always visible by default
The grid toggle now uses a robust add/remove approach instead of simple visibility toggling:
// Reliable toggle using scene add/remove
if (debugMesh.parent) {
// Hide: Remove from scene
debugMesh.parent.remove(debugMesh);
gridToggleButton.textContent = 'Show Grid';
} else {
// Show: Add back to scene
terrainScene.add(debugMesh);
gridToggleButton.textContent = 'Hide Grid';
}
Multiple improvements create more engaging physics demonstrations:
The system uses multiple smaller substeps for accurate collision detection without sacrificing performance:
// Multiple substeps for accuracy
const substeps = 2;
const substepTime = deltaTime / substeps;
for (let i = 0; i < substeps; i++) {
world.timestep = substepTime;
world.step();
}
CCD prevents fast-moving objects from tunneling through terrain:
// Enable CCD for dynamic bodies
if (mass > 0) {
desc.setCcdEnabled(true);
}
The critical connection between visual terrain and physics simulation:
// Extract height data in correct format for Rapier
for (let z = 0; z < depth; z++) {
for (let x = 0; x < width; x++) {
const vertexIndex = (z * width + x) * 3;
// Z component contains height before terrain rotation
const height = positions[vertexIndex + 2];
heightData[z * width + x] = height;
}
}
Ensuring the debug grid perfectly matches the physics collision surface:
// Use exact terrain geometry for debug visualization
const debugGeometry = terrainGeometry.clone();
// Apply identical transformations
debugMesh.position.copy(terrainMesh.position);
debugMesh.rotation.copy(terrainMesh.rotation);
debugMesh.scale.copy(terrainMesh.scale);
// Add to same scene for consistent transformation
terrainScene.add(debugMesh);
Before Improvements:
After Improvements:
This terrain physics demonstration showcases how careful integration of modern web technologies can create compelling, realistic physics simulations in the browser. The key to success lies in:
The resulting system provides a solid foundation for more complex physics simulations and demonstrates best practices for web-based 3D physics development. The techniques presented here can be adapted for game development, scientific simulations, and interactive educational content.
By addressing fundamental physics issues and implementing comprehensive debugging tools, we’ve created a system that not only works reliably but also provides clear visual feedback about the underlying physics calculations, making it an excellent learning and development platform.
How to properly integrate WASM-based physics into modern web games without falling into common pitfalls
When building RedactedProjectName, a game with TypeScript and Three.js, I encountered significant challenges integrating Rapier physics. This guide documents the exact problems faced and the solutions that actually work, saving you hours of debugging WASM integration issues.
Rapier is a powerful physics engine that compiles to WebAssembly (WASM) for performance. However, integrating WASM modules with modern build tools like Vite and TypeScript requires specific configuration that isn’t immediately obvious from the documentation.
❌ Wrong Approach: Creating placeholder/fallback systems
When I first encountered WASM loading issues, my instinct was to create a placeholder physics system and defer the “real” integration. This is a common anti-pattern that obscures the root cause.
// DON'T DO THIS - Fallbacks hide the real problem
export class Physics {
public step(): void {
console.log('Physics placeholder - will implement later');
// This never gets properly implemented
}
}
✅ Right Approach: Address the root cause immediately
The real issue wasn’t complexity—it was missing Vite configuration for WASM handling.
❌ Wrong Approach: Static imports
// This fails in Vite with WASM modules
import RAPIER from '@dimforge/rapier3d';
export class Physics {
constructor() {
// This will throw errors about WASM loading
const world = new RAPIER.World({ x: 0, y: -9.81, z: 0 });
}
}
✅ Right Approach: Dynamic imports with proper async handling
// This works correctly
export class Physics {
private RAPIER: typeof import('@dimforge/rapier3d') | null = null;
private world: import('@dimforge/rapier3d').World | null = null;
constructor() {
this.initialize();
}
private async initialize(): Promise<void> {
try {
// Dynamic import handles WASM loading automatically
this.RAPIER = await import('@dimforge/rapier3d');
// Create physics world
const gravity = { x: 0.0, y: -9.81, z: 0.0 };
this.world = new this.RAPIER.World(gravity);
console.log('⚡ Physics initialized successfully');
} catch (error) {
console.error('Failed to initialize physics:', error);
}
}
}
The critical missing piece was proper Vite configuration for WASM handling.
npm install --save-dev vite-plugin-wasm vite-plugin-top-level-await
// vite.config.ts
import { defineConfig } from 'vite';
import wasm from 'vite-plugin-wasm';
import topLevelAwait from 'vite-plugin-top-level-await';
export default defineConfig({
plugins: [
wasm(), // Handles WASM file loading
topLevelAwait() // Enables top-level await for WASM
],
build: {
rollupOptions: {
output: {
manualChunks: {
'physics': ['@dimforge/rapier3d'], // Separate chunk for physics
}
}
}
},
optimizeDeps: {
exclude: [
'@dimforge/rapier3d' // Don't pre-bundle WASM modules
]
}
});
❌ Wrong Approach: Using any types everywhere
// Loses all type safety
private world: any = null;
private bodies: Map<string, any> = new Map();
✅ Right Approach: Proper TypeScript integration
// Maintains full type safety
type RAPIER = typeof import('@dimforge/rapier3d');
type World = import('@dimforge/rapier3d').World;
type RigidBody = import('@dimforge/rapier3d').RigidBody;
export class Physics {
private RAPIER: RAPIER | null = null;
private world: World | null = null;
private bodies: Map<string, RigidBody> = new Map();
public createDynamicBody(
id: string,
position: THREE.Vector3,
shape: 'box' | 'sphere',
size: THREE.Vector3 | number
): RigidBody | null {
if (!this.world || !this.RAPIER) return null;
const bodyDesc = this.RAPIER.RigidBodyDesc.dynamic()
.setTranslation(position.x, position.y, position.z);
let colliderDesc: import('@dimforge/rapier3d').ColliderDesc;
switch (shape) {
case 'box':
const boxSize = size as THREE.Vector3;
colliderDesc = this.RAPIER.ColliderDesc.cuboid(
boxSize.x / 2, boxSize.y / 2, boxSize.z / 2
);
break;
case 'sphere':
const radius = size as number;
colliderDesc = this.RAPIER.ColliderDesc.ball(radius);
break;
}
const rigidBody = this.world.createRigidBody(bodyDesc);
this.world.createCollider(colliderDesc, rigidBody);
this.bodies.set(id, rigidBody);
return rigidBody;
}
}
One of the most frustrating aspects of WASM integration is that development and production builds behave differently.
# Test development
npm run dev
# Test production build
npm run build
npm run preview
Important: Always test your WASM integration in production mode before deploying!
❌ Wrong Approach: Assuming synchronous initialization
// This fails because physics isn't ready yet
constructor() {
this.physics = new Physics();
this.createPhysicsObjects(); // ERROR: Physics not initialized
}
✅ Right Approach: Proper async initialization handling
export class Engine {
private setupPhysicsDemo(): void {
const checkPhysics = () => {
if (this.physics.isReady()) {
this.scene.createPhysicsCube(this.physics);
console.log('Physics demo ready!');
} else {
// Check again in 100ms
setTimeout(checkPhysics, 100);
}
};
checkPhysics();
}
}
export class Physics {
public isReady(): boolean {
return this.isInitialized && this.world !== null;
}
}
Here’s a minimal but complete example that demonstrates all the concepts:
{
"dependencies": {
"@dimforge/rapier3d": "^0.11.2",
"three": "^0.158.0"
},
"devDependencies": {
"vite": "^4.4.5",
"vite-plugin-wasm": "^3.5.0",
"vite-plugin-top-level-await": "^1.6.0",
"typescript": "^5.0.2"
}
}
import * as THREE from 'three';
type RAPIER = typeof import('@dimforge/rapier3d');
type World = import('@dimforge/rapier3d').World;
type RigidBody = import('@dimforge/rapier3d').RigidBody;
export class Physics {
private RAPIER: RAPIER | null = null;
private world: World | null = null;
private isInitialized = false;
constructor() {
this.initialize();
}
private async initialize(): Promise<void> {
try {
this.RAPIER = await import('@dimforge/rapier3d');
const gravity = { x: 0.0, y: -9.81, z: 0.0 };
this.world = new this.RAPIER.World(gravity);
this.isInitialized = true;
console.log('⚡ Physics initialized');
} catch (error) {
console.error('Physics initialization failed:', error);
}
}
public isReady(): boolean {
return this.isInitialized && this.world !== null;
}
public step(): void {
if (this.world) {
this.world.step();
}
}
public createDynamicBox(
position: THREE.Vector3,
size: THREE.Vector3
): RigidBody | null {
if (!this.world || !this.RAPIER) return null;
const bodyDesc = this.RAPIER.RigidBodyDesc.dynamic()
.setTranslation(position.x, position.y, position.z);
const colliderDesc = this.RAPIER.ColliderDesc.cuboid(
size.x / 2, size.y / 2, size.z / 2
);
const rigidBody = this.world.createRigidBody(bodyDesc);
this.world.createCollider(colliderDesc, rigidBody);
return rigidBody;
}
}
When properly configured, you should see output like this:
✓ 280 modules transformed.
dist/assets/rapier_wasm3d_bg-a8e9a6c4.wasm 1,409.61 kB
dist/assets/physics-8c074953.js 145.88 kB │ gzip: 23.97 kB
dist/assets/three-f2ff3508.js 543.34 kB │ gzip: 121.41 kB
✓ built in 1.32s
The key indicators of success:
vite-plugin-wasm is essential// Add this to verify WASM is loading
private async initialize(): Promise<void> {
console.log('Starting Rapier initialization...');
try {
const start = performance.now();
this.RAPIER = await import('@dimforge/rapier3d');
const loadTime = performance.now() - start;
console.log(`Rapier loaded in ${loadTime.toFixed(2)}ms`);
this.world = new this.RAPIER.World({ x: 0, y: -9.81, z: 0 });
console.log('Physics world created successfully');
} catch (error) {
console.error('Detailed error:', error);
console.error('Error stack:', error.stack);
}
}
In browser dev tools, check that:
Integrating Rapier physics with TypeScript and Vite requires specific configuration, but once properly set up, it provides excellent performance and developer experience. The key is avoiding fallback patterns and addressing WASM integration directly with the right tools.
With these patterns, you can confidently integrate Rapier physics into any TypeScript web project without the common pitfalls that plague WASM integration.
Have questions or improvements? Email me at andy@greenrobot.com