# Shashwat Dixit > Software Engineer building performant backends and full-stack applications. I care about systems that scale and developer experience that doesn't suck. Bengaluru, India. [Website](https://shashwatdixit.com) · [Email](mailto:shashwatmain@gmail.com) ## About I'm a software engineer at [Interview Kickstart](https://interviewkickstart.com) working on payments — gateway integrations, installment billing, and the flow that grants learners access after they pay. Previously at [Instahyre](https://instahyre.com) I built distributed systems, search infrastructure, and optimized backend performance. Before that I built event-driven pipelines and SSR frontends as a [full-stack contractor](/#work). I hold a degree in [Electrical & Electronics Engineering from NMIT Bengaluru](https://nmit.ac.in) and have authored [IEEE research papers on Quantum Computing](https://scholar.google.com/citations?user=q3MbjLQAAAAJ&hl=en). I like working across the stack — from Redis locking to Elasticsearch query parsers to Next.js SPAs. ## Work ### [Interview Kickstart](https://interviewkickstart.com) — SDE-1 Bengaluru, India. June 2026 – Present - Fixed 5 payment-gateway defects against the live API format, stopping real transactions from silently falling back to a backup provider - Fixed a double-counting bug in installment balance logic that was silently rejecting valid payments - Shipped a feature-flagged payment-flow rewrite with old and new systems running side-by-side for instant rollback - Cut learner activation delay by granting access on payment completion, with a 30-day auto-revoke if payment is never confirmed - Cut error-monitoring noise by ~30% by filtering expected no-ops, and added reference-ID logging for faster debugging ### [Instahyre](https://instahyre.com) — SDE-1 Bengaluru, India. December 2024 – May 2026 - Designed Redis-based distributed locking (SETNX + TTL) to eliminate double-booking under concurrent traffic - Reduced API latency by 80% (p75: 600ms → 120ms) by profiling slow PostgreSQL queries and optimizing payloads - Built a stack-based boolean query parser (AND/OR/NOT) for Elasticsearch, improving matching relevance by 20% - Reduced regression bugs by 30% by decomposing a monolith into independently maintainable modules ### [Freelance Work](https://upwork.com) — Full-Stack Developer Remote. May 2024 – August 2024 - Designed an event-driven pipeline using Kafka for asynchronous order processing across thousands of daily events - Implemented idempotent webhook-based payment confirmation, reducing duplicate order processing by 95% - Improved Core Web Vitals by 35% with a Next.js SSR architecture optimized for first-contentful paint and reduced JS bundle size ## Education ### [Nitte Meenakshi Institute of Technology, Bengaluru](https://nmit.ac.in) Bachelor of Engineering in Electrical & Electronics Engineering. 2020 – 2024 ## Skills Python, C++, Go, JavaScript, TypeScript, Git, GCP, BigQuery, AWS, Docker, Kubernetes, React, Next.js, Node.js, Bun.js, Elasticsearch, Django, RabbitMQ, Kafka, PostgreSQL, Redis, Cassandra, MySQL, SQL ## Projects ### [Jamin](https://jamin.shashwatdixit.com) Full-stack AI chat platform integrating multiple LLMs (GPT, Claude, Cohere) via LangChain. Features RAG-powered PDF Q&A and YouTube summarization, plus Stable Diffusion image generation. Scalable PostgreSQL/Drizzle backend enabling context-aware conversations across models. Tech: React, Node.js, LangChain, PostgreSQL, Drizzle, OpenAI, Stable Diffusion - [Source](https://github.com/shashwat-dixit/jamin) ### [Zort](https://zort.shashwatdixit.com) Real-time collaborative whiteboard supporting 100+ concurrent users, built with Next.js and Socket.IO. Client-side state management via Zustand and localStorage to reduce server load. CI/CD pipeline with Docker and GitHub Actions for zero-failure automated deployments. Tech: Next.js, Socket.IO, Zustand, Docker, GitHub Actions - [Source](https://github.com/shashwat-dixit/zort) ### [Phabric](https://phabric.shashwatdixit.com) Phabric is Vercel but with websocket support! Tech: Next.js, Socket.IO, Zustand, Docker, GitHub Actions - [Source](https://github.com/shashwat-dixit/zort) ### [Code Compete](https://codecompete.shashwatdixit.com) Code Compete is a platform for competitive programming problems. It is built with Next.js and Tailwind CSS. It is a platform for competitive programming problems. It is a platform for competitive programming problems. Tech: Next.js, Socket.IO, Zustand, Docker, GitHub Actions - [Source](https://github.com/shashwat-dixit/zort) ## Contact - Email: shashwatmain@gmail.com - [GitHub](https://github.com/shashwat-dixit) - [LinkedIn](https://linkedin.com/in/dixitshashwat) - [X](https://x.com/shashwatmain) Markdown index for agents: https://shashwatdixit.com/llms.txt --- --- title: "What I Learned Shipping 300 Code Reviews in 1.5 Years as a Fresher" slug: what-300-code-reviews-taught-me date: 2026-01-05 updated: 2026-01-05 tags: - code description: "From fixing undefined variables to decomposing a monolith — a career progression compressed into data." status: published --- *From fixing undefined variables to decomposing a monolith — a career progression compressed into data.* --- ## The Numbers In 18 months as an IC-1 (fresher-level individual contributor), I authored 302 revisions on Phabricator: - **~200 published** (landed in production) - **~80 abandoned** (scrapped, restarted, superseded) - **~20 in other states** (needs review, draft) That's roughly 2.5 published revisions per week, sustained over 1.5 years. But the numbers alone don't tell the story. The *type* of work changed dramatically from month 1 to month 18. --- ## Phase 1: The Bug Fix Grind (Months 1-6) My first 100+ revisions were almost entirely bug fixes: ``` D27158: Fixes Undefined var TypeError: primarySkills is undefined D27181: Fix no space between text area and rating symbols D27293: Fixed undefined var TypeError: $element[0] is undefined D27295: Fixes Undefined var Error: filters is undefined D27310: Fixes Resume modal: No space after comma D27370: Fixes Undefined var Error: action_stages is undefined D27398: Fixed tooltip not showing for request evaluation ``` These look small. They are small. But here's what they taught me: ### What I Learned From Bug Fixes 1. **How to read a stack trace.** Sentry errors with JS undefined vars forced me to trace execution flow through AngularJS controllers and services. 2. **How the codebase is structured.** Every bug fix required understanding which controller owns which scope variable, which service fetches which data, which template renders which state. 3. **How to write a good revision summary.** Early on, my summaries were "Fix bug." By month 3, they included: what the bug was, what caused it, what I changed, and how to test it. 4. **How to handle the review process.** My first revisions had 3-4 rounds of review feedback. By month 4, most landed in 1-2 rounds. 5. **Pattern recognition.** After fixing 20 undefined variable bugs, I started seeing the anti-patterns that produced them. This became useful later when I started writing features. ### The Uncomfortable Truth Bug fixes are not glamorous. Nobody's impressed by "Fixed undefined var." But they're the fastest way to build: - Codebase familiarity - Trust from the team - Confidence to make larger changes --- ## Phase 2: Small Features and Polish (Months 6-10) Around month 6, my revisions shifted: ``` D29044: Fix cancel evaluation modal width and height issue D29884: Fix evaluation bugs for interviewer role D30267: Fix evaluation tab count issue changing issue on applying filters D30401: Add graceful handling for profile type mismatch D30483: Simplify evaluation count logic in templates D31016: Return Credit Info and Failure Message on Bulk Action Errors ``` The last one — `Return Credit Info and Failure Message on Bulk Action Errors` — was my first "feature" revision. Not a bug fix. Not a UI tweak. A deliberate backend enhancement that required designing a response format and coordinating with the frontend. ### What Changed - I started proposing solutions instead of being assigned bugs - I began touching the Python/Django layer, not just JS/CSS - I started owning small end-to-end flows (API → frontend → user message) --- ## Phase 3: Architecture and Performance (Months 10-18) This is where the work got interesting: ``` D28249: Separate Python Code for Evaluations D28269: Separated CSS Code For Evaluations D28365: Separate JS Code For Evaluations D28367: Separate HTML Code for Evaluations D28427: merge code-separation into master D34678: Add Custom Browser Find Feature D35117: Optimize Queries in get_candidate_stages API D36701: Log function arguments for celery tasks D36704: Restrict registering app APIs into base v1 API D37393: Improve performance of GET /candidate/{identifier} ``` I was now: - **Decomposing the monolith** — multi-revision architectural changes - **Optimizing queries** — reading EXPLAIN plans, adding select_related - **Building features from scratch** — the custom Ctrl+F search - **Writing infrastructure tooling** — lint rules, logging - **Documenting methodology** — Phriction docs for the team --- ## The Abandoned Revisions 80 out of 302 revisions were abandoned. Early on, this felt like failure. Now I see it differently. Common reasons for abandonment: ### 1. Wrong approach discovered during review ``` D37385: Improve performance of GET /candidate/{identifier} - API Group (Abandoned → superseded by D37393 with a better approach) ``` I'd write a solution, get review feedback that a different approach was better, and start fresh instead of frankensteining the original. ### 2. Exploratory revisions ``` D35740: Logging for T43722 D35011: Logging for T42059 ``` These were diagnostic revisions — add logging, observe production behavior, remove logging. The logging itself was never meant to ship permanently. ### 3. Scope too large, needed to split ``` D34541: Move all template logic to JS Code for Easier Debugging (Abandoned → broken into smaller focused revisions) ``` ### 4. Genuine mistakes ``` D36712: Test linter added in D36704 (Oops, committed a test revision) ``` ### The Lesson **Abandoned revisions are cheap. Bad code in production is expensive.** If your alternative to abandoning is shipping something you're not confident in, abandon every time. The best engineers I work with have high abandon rates because they iterate aggressively. --- ## What I Wish I'd Known on Day 1 ### 1. Read the codebase before writing code My first few revisions had embarrassing mistakes — reimplementing things that already existed, using patterns inconsistent with the rest of the code. Spend your first week reading, not writing. ### 2. Small revisions > large revisions My monolith decomposition landed as 5 separate revisions. If I'd tried to do it in 1 massive diff: - Reviewers wouldn't have reviewed it carefully - A revert would have been all-or-nothing - I couldn't have gotten incremental feedback Target 100-300 lines per revision. If it's bigger, ask: "Can this be split?" ### 3. Write the summary as if the reviewer knows nothing ``` # Bad summary Fix the bug # Good summary Fix: Evaluation count shows wrong value on search page after adding note Bug: After adding a note from the search page, the evaluation count badge increments by 1 even though no evaluation was submitted. Cause: The note-creation event triggers the same event listener that evaluation submission uses. The listener doesn't distinguish between event types. Fix: Add event type check in the listener. Only increment count when the event type is 'evaluation_submitted'. Test: Add note on search page → count stays the same. ``` The second version reviews itself. The reviewer can verify the fix just by reading the summary. ### 4. Understand the system, not just the symptom My best work came from asking "why does this pattern exist?" rather than "how do I fix this instance?" - Seeing repeated undefined var bugs → led me to understand AngularJS scope inheritance - Seeing repeated CSS conflicts → led me to propose app-scoped CSS - Seeing repeated API namespace collisions → led me to write the lint rule ### 5. Document what you learn I wrote two internal docs: - "How to Run Compress Locally" — because I spent 2 hours figuring it out and didn't want anyone else to - "How to Separate an App" — because the monolith decomposition process wasn't written down anywhere These took 30 minutes each. They saved hours for every new hire after me. --- ## The Progression, Visualized ``` Month 1-3: ████████████████████████ Bug fixes (undefined vars, CSS) Month 4-6: ████████████░░░░░░░░░░░░ Bug fixes + UI polish Month 7-9: ████████░░░░░░░░████████ Features + App separation Month 10-12: ████░░░░░░░░████████████ Performance + Architecture Month 13-18: ██░░░░████████████████░░ Architecture + Infra + Features ``` The ratio of "fix someone else's bug" to "build something new" steadily inverted. --- ## Metrics That Actually Matter Looking back, these are the metrics that correlated with real growth: | Metric | Why It Matters | |--------|---------------| | **Revisions per week** | Consistency > bursts. 2-3/week means you're reliably shipping. | | **Lines of code per revision** | Smaller = better reviewed = higher quality. | | **Review rounds before landing** | Decreasing over time means you're internalizing the codebase standards. | | **Scope of changes** | Are you touching 1 file or 10? Single-layer or full-stack? | | **Who assigns your work** | Self-assigned > assigned. It means you're spotting problems. | | **Abandoned-to-published ratio** | A healthy ratio (~20-30%) means you're experimenting, not just playing safe. | --- ## Advice for Other Freshers 1. **The bug fix phase is an investment, not a punishment.** You're building a mental model of the system. This pays off exponentially when you start building features. 2. **Ship frequently, not perfectly.** A revision that ships with one round of feedback is better than a "perfect" revision that takes a week. 3. **Go beyond your ticket.** If you fix a bug and notice two more, fix those too. If you see a pattern, propose a systemic fix. This is how you get trusted with architecture work. 4. **Read other people's revisions.** I learned more from reviewing my teammates' code than from writing my own. You see patterns, approaches, and mistakes you can avoid. 5. **Own a module.** After 6 months, I became "the evaluations person." This wasn't assigned — I just kept fixing evaluations bugs until I understood it better than anyone. Then the decomposition work was a natural next step. 6. **Write things down.** Two 30-minute docs earned me more visibility than 50 bug fixes. Documentation signals that you're thinking about the team, not just your own output. --- ## Where It Led After 18 months: - Owned the full evaluations module end-to-end - Optimized 4 high-traffic API endpoints - Built architectural tooling (lint rules, logging) - Documented repeatable engineering processes - Contributed to the Python 3 migration From a career positioning standpoint, this body of work is equivalent to what many engineers accumulate in 3-4 years — because the volume was high, the scope expanded continuously, and I documented everything. The 302 revisions aren't the point. The arc from "fix undefined variable" to "decompose the monolith" — that's the story. --- --- title: "Optimizing Django ORM Queries" slug: django-orm-query-optimization date: 2025-11-29 updated: 2025-11-29 tags: - code description: "Practical patterns for eliminating N+1 queries, reducing p90 latency, and knowing when to split an API endpoint." status: published --- *Practical patterns for eliminating N+1 queries, reducing p90 latency, and knowing when to split an API endpoint.* --- ## Introduction I spent a significant chunk of my first 1.5 years as a backend engineer optimizing slow API endpoints in a Django application. The product is an ATS (Applicant Tracking System) where recruiters view candidate profiles, stages, events, and evaluations — all loaded via REST APIs. Our monitoring showed several endpoints with p90 latencies exceeding 2 seconds. After investigating, the root causes were almost always the same patterns: 1. **N+1 queries** — Fetching related objects inside a loop 2. **Redundant queries** — Fetching the same data multiple times 3. **Bloated endpoints** — One API doing too much, blocking page load 4. **Missing aggregation** — Using Python to compute what the database can do This post covers the patterns I applied repeatedly, with real (anonymized) examples. --- ## Pattern 1: select_related for Foreign Keys ### The Problem ```python # Fetching candidate opportunities opportunities = Opportunity.objects.filter(candidate_id=can_id) for opp in opportunities: job_title = opp.job.title # Query per iteration employer_name = opp.job.employer.name # Another query per iteration stage_name = opp.action_stage.name # Yet another query ``` For a candidate with 10 opportunities, this fires 30+ queries instead of 1. ### The Fix ```python candidate_opps = list( Opportunity.objects.filter(candidate_id=can_id) .select_related('job', 'job__employer', 'action_stage', 'candidate') .distinct() ) # Now all access is from memory — zero additional queries for opp in candidate_opps: job_title = opp.job.title employer_name = opp.job.employer.name stage_name = opp.action_stage.name ``` `select_related` works by doing a SQL JOIN. For our case, the single query with JOINs was ~5ms vs the original ~150ms from 30 round-trips. ### When to Use - **One-to-one** or **many-to-one** (ForeignKey) relationships - You know you'll access the related object for most/all items - The related table isn't enormous (JOINs on billion-row tables have their own issues) ### When NOT to Use - **Many-to-many** or **reverse foreign keys** — use `prefetch_related` instead - You only need the related object's ID (it's already on the FK field: `opp.job_id`) --- ## Pattern 2: Pre-fetching to Eliminate N+1 in Business Logic ### The Problem Sometimes the N+1 isn't in the ORM layer — it's in business logic: ```python for opp in candidate_opps: job_id = opp.job_id # This hits the cache or DB for each job if job_id not in recruiter.subscribed_job_ids: owned_stages = recruiter.get_owned_stages_for_job(job_id) # This queries events table for each opportunity upcoming_event = get_upcoming_event(opp.id, recruiter.id) ``` ### The Fix: Pre-fetch Outside the Loop ```python # Pre-fetch owned stages for ALL jobs in one pass job_ids = list(set(opp.job_id for opp in candidate_opps)) subscribed_job_ids = set(recruiter.subscribed_job_ids) owned_stages_map = {} for job_id in job_ids: if job_id not in subscribed_job_ids: owned_stages_map[job_id] = \ recruiter.get_owned_stages_position_for_job(job_id) # Pre-fetch ALL upcoming events for this recruiter in one query opp_ids = set(opp.id for opp in candidate_opps) all_upcoming_events = list( Event.objects.filter( opportunity_id__in=opp_ids, participants=recruiter, start_time__gt=timezone.now() ).select_related('opportunity') ) # Build a lookup dict events_by_opp = defaultdict(list) for event in all_upcoming_events: events_by_opp[event.opportunity_id].append(event) # Now the loop is O(1) lookups for opp in candidate_opps: owned_stages = owned_stages_map.get(opp.job_id, []) upcoming = events_by_opp.get(opp.id, []) ``` This transformed a 15-query loop into 2 queries + dictionary lookups. --- ## Pattern 3: Aggregation Instead of Multiple Queries ### The Problem We needed to check if a candidate was shared via email OR via excel sheet: ```python # Original: Two separate queries shared_via_email = ProfileShare.objects.filter( candidates=can_id, job_id=job_id, source=ProfileShare.SHARE_ACTION, to_emails__contains=recruiter_email ).exists() shared_via_excel = ProfileShare.objects.filter( candidates=can_id, job_id=job_id, source=ProfileShare.EXCEL_SHEET, job__employer=employer ).exists() ``` Two database round-trips for something the DB can answer in one. ### The Fix: Single Query with Case/When Aggregation ```python from django.db.models import Case, When, Value, IntegerField, Max shares = ProfileShare.objects.filter( job__is_active=True, created_at__gte=last_30_days, candidates=can_id, job_id=job_id ) result = shares.aggregate( has_email_share=Max( Case( When(shared_mail_query, then=Value(1)), default=Value(0), output_field=IntegerField() ) ), has_excel_share=Max( Case( When(shared_via_excel_query, then=Value(1)), default=Value(0), output_field=IntegerField() ) ) ) shared_with_email = bool(result.get('has_email_share')) shared_via_excel = bool(result.get('has_excel_share')) exists = shared_with_email or shared_via_excel ``` One query. The database evaluates both conditions in a single table scan. ### When to Use - You need to check multiple conditions on the same queryset - You need counts/existence checks across categories - The alternative is multiple `.filter().exists()` or `.filter().count()` calls --- ## Pattern 4: Reusing Already-Fetched Objects ### The Problem ```python opp = Opportunity.objects.get(id=opp_id) # Later in the same view... job = Job.objects.get(id=opp.job_id) # Unnecessary — we could have JOINed ``` ### The Fix ```python opp = Opportunity.objects.select_related( 'candidate', 'candidate__job_search_preferences', 'candidate__user', 'job', 'job__employer', 'action_stage', ).get(id=opp_id) # Reuse the already-loaded relation job = opp.job # No query — loaded via select_related ``` This sounds obvious, but in a large codebase with many contributors, it's common for code added later to re-fetch objects that are already available. A comment helps: ```python # Reuse opp.job (loaded via select_related) when available, # fall back to separate query otherwise. if opp is not None: job = opp.job else: job = Job.objects.get(id=job_id) ``` --- ## Pattern 5: Splitting Heavy Endpoints ### The Problem Our candidate profile page loaded everything in one API call: - Candidate info - Job stages - Unread messages count - Starred messages - Upcoming events The unread/starred messages query was slow (scanning a large messages table with complex filters), and it blocked the entire page from rendering. ### The Fix: Separate Endpoint ```python # Before: Everything in SingleOpportunityResource.get_detail() def get_detail(self, request): data = self.get_candidate_info() data['stages'] = self.get_stages() data['unread_count'] = self.get_unread_messages() # SLOW data['starred'] = self.get_starred_messages() # SLOW data['events'] = self.get_upcoming_events() return data # After: Messages moved to their own endpoint # GET /api/v1/candidate/messages_status/?candidate_id=123 class MessagesStatusResource(Resource): def get_detail(self, request): return { 'unread_count': self.get_unread_messages(), 'starred': self.get_starred_messages(), } ``` The frontend now fires both requests in parallel: ```javascript // Load in parallel — page renders as soon as the fast one returns $q.all([ singleOpportunityService.getCandidateProfile(candidateId), messagesService.getMessagesStatus(candidateId) ]).then(function([profile, messages]) { $scope.profile = profile; $scope.unreadCount = messages.unread_count; }); ``` ### When to Split - One sub-query is significantly slower than the rest - The slow data isn't needed for initial render - The frontend can progressively load the data - The slow query operates on a different table/index --- ## Pattern 6: Fixing LEFT OUTER JOINs ### The Problem A query to check if a candidate had an interview scheduled: ```python # This generates a LEFT OUTER JOIN on the events table interviews = Opportunity.objects.filter( candidate_id=can_id ).filter( events__event_type='interview', events__status='scheduled' ) ``` LEFT OUTER JOINs are expensive when the right table (events) is large and doesn't have the right composite index. ### The Fix: Subquery or Exists ```python from django.db.models import Exists, OuterRef # Subquery approach — generates EXISTS instead of JOIN has_interview = Event.objects.filter( opportunity=OuterRef('pk'), event_type='interview', status='scheduled' ) interviews = Opportunity.objects.filter( candidate_id=can_id ).annotate( has_scheduled_interview=Exists(has_interview) ).filter( has_scheduled_interview=True ) ``` `EXISTS` stops scanning the events table at the first match. A JOIN loads all matching rows. --- ## Measuring Impact Before and after each optimization, I checked: 1. **Query count** — Django Debug Toolbar or `connection.queries` 2. **p50/p90/p99** — From our APM tool (Datadog) 3. **Database time** — Sum of all query durations in the request A typical result: | Metric | Before | After | |--------|--------|-------| | Queries per request | 35 | 8 | | p50 latency | 450ms | 120ms | | p90 latency | 2100ms | 350ms | | DB time | 380ms | 85ms | --- ## Debugging Tools 1. **Django Debug Toolbar** — Shows all queries, duplicates highlighted 2. **`connection.queries`** — Programmatic access to query log in development 3. **`EXPLAIN ANALYZE`** — Run the raw SQL in MySQL/PostgreSQL to see the execution plan 4. **APM (Datadog/New Relic)** — Production p90/p99 percentiles 5. **Silk** — Django middleware that profiles requests and stores results Quick snippet to log queries in development: ```python from django.db import connection, reset_queries import functools def query_debugger(func): @functools.wraps(func) def wrapper(*args, **kwargs): reset_queries() result = func(*args, **kwargs) queries = connection.queries print(f"Function: {func.__name__}") print(f"Number of Queries: {len(queries)}") print(f"Total Time: {sum(float(q['time']) for q in queries):.3f}s") return result return wrapper ``` --- ## Summary of Patterns | Pattern | When to Use | Typical Savings | |---------|-------------|-----------------| | `select_related` | FK/OneToOne access in loops | 5-50x fewer queries | | Pre-fetch outside loop | Business logic N+1 | 3-20x fewer queries | | Aggregate with Case/When | Multiple exists/count checks | 2-5x fewer queries | | Reuse loaded objects | Same relation accessed twice | 1 fewer query per occurrence | | Split endpoint | One slow sub-query blocks render | 50-80% faster perceived load | | EXISTS instead of JOIN | Checking existence, not fetching data | 2-10x on large tables | The key mindset shift: **think in sets, not loops.** Every time you write a `for` loop that accesses a related object, ask: "Can I fetch all of these in one query before the loop?" --- --- title: "Custom Ctrl f For Embedded Content" slug: custom-search-for-embedded-content date: 2025-10-11 updated: 2025-10-11 tags: - code description: "How to intercept the browser's native find dialog and implement scoped search within" status: published --- *How to intercept the browser's native find dialog and implement scoped search within an embedded document viewer.* --- ## The Problem Our app displays candidate resumes inside a modal. The resume is rendered either as a PDF (via pdf.js) or as parsed HTML inside a container. The problem: when a recruiter presses Ctrl+F to search for a keyword in the resume, the browser's native find dialog opens and searches the **entire page** — including navigation, sidebars, other tabs, and background content. This is terrible UX. The recruiter wants to find "Python" in the resume, not in the sidebar menu or in a hidden tab's content. --- ## The Solution: Intercept and Replace We need to: 1. Intercept Ctrl+F when the resume tab is active 2. Prevent the native find dialog from opening 3. Show a custom search bar scoped to the resume content 4. Highlight matches within the PDF/HTML text layer 5. Support Ctrl+G / Enter for "find next" navigation 6. Handle accented characters (Résumé → Resume should match) --- ## Implementation ### Step 1: Intercept Ctrl+F The key insight: `event.preventDefault()` on the `keydown` event stops the native find dialog, but only if you catch it before the browser processes it. ```javascript var handleKeydown = function(event) { if ((event.ctrlKey || event.metaKey) && event.key === "f") { // Only intercept when resume tab is active if (isResumeTabActive()) { event.preventDefault(); event.stopPropagation(); // Focus our custom search input var input = document.getElementById("resume-search-input"); if (input) { input.focus(); input.select(); } } } }; document.addEventListener("keydown", handleKeydown); ``` Important: use `event.key === "f"` rather than checking `keyCode`. It handles keyboard layouts correctly and is the modern standard. ### Step 2: The Search Bar UI ```html
{{ currentMatchIndex }} / {{ totalMatches }}
``` ### Step 3: Finding Matches in the Text Layer PDF.js renders text in a `
` with individual `` elements for each text segment. HTML resumes use a similar structure. We search within these spans: ```javascript function findAndHighlightMatches(searchTerm) { var matches = []; var normalizedSearch = normalizeText(searchTerm); if (normalizedSearch.length === 0) return matches; // Target both PDF text layer and HTML resume container var textSpans = document.querySelectorAll( "#pdfjs-container .textLayer span:not(.highlight), " + "#resume-html-container .textLayer span:not(.highlight)" ); textSpans.forEach(function(span) { var text = span.textContent; var normalizedText = normalizeText(text); if (normalizedText.indexOf(normalizedSearch) === -1) { return; // Skip spans without matches } // Store original content for later restoration if (!originalContents.has(span)) { originalContents.set(span, span.innerHTML); } // Find all positions of the search term var matchPositions = []; var idx = 0; while ((idx = normalizedText.indexOf(normalizedSearch, idx)) !== -1) { matchPositions.push({ start: idx, end: idx + normalizedSearch.length }); idx += normalizedSearch.length; } // Build highlighted HTML var result = ""; var lastEnd = 0; matchPositions.forEach(function(pos) { var origStart = mapNormalizedToOriginal(text, pos.start); var origEnd = mapNormalizedToOriginal(text, pos.end); result += escapeHtml(text.substring(lastEnd, origStart)); result += "" + escapeHtml(text.substring(origStart, origEnd)) + ""; matches.push({ element: span, matchIndex: matches.length }); lastEnd = origEnd; }); result += escapeHtml(text.substring(lastEnd)); span.innerHTML = result; }); return matches; } ``` ### Step 4: Accent-Insensitive Search Resumes often contain accented characters (especially names). "José" should match when you search for "Jose". We use Unicode NFD normalization: ```javascript function normalizeText(text) { return text.normalize("NFD") .replace(/[\u0300-\u036f]/g, "") // Strip combining marks .toLowerCase(); } ``` But here's the tricky part: when we normalize text for searching, the character positions shift. "Résumé" normalizes to "resume" but the accent characters occupy positions in the original string. We need a mapping function: ```javascript function mapNormalizedToOriginal(originalText, normalizedPos) { var origIndex = 0; var normIndex = 0; while (normIndex < normalizedPos && origIndex < originalText.length) { var char = originalText[origIndex]; var normalized = char.normalize("NFD") .replace(/[\u0300-\u036f]/g, ""); if (normalized.length === 0) { // This is a combining mark — skip in original, don't advance // normalized position origIndex++; } else { normIndex++; origIndex++; } } return origIndex; } ``` This walks through both the original and normalized text in lockstep, correctly mapping positions even when accented characters expand or contract. ### Step 5: Match Navigation (Ctrl+G / Enter) ```javascript // Ctrl+G for next match (mirrors Chrome's behavior) var handleCtrlG = function(event) { if ((event.ctrlKey || event.metaKey) && event.key === "g") { if (isResumeTabActive()) { event.preventDefault(); event.stopPropagation(); var input = document.getElementById("resume-search-input"); if (!input.value) { input.focus(); input.select(); } else { goToNextMatch(); } } } }; // Enter/Shift+Enter on the search input var handleSearchKeydown = function(event) { if (event.key === "Enter" && isSearchInputFocused()) { event.preventDefault(); if (event.shiftKey) { goToPrevMatch(); } else { goToNextMatch(); } } }; ``` ### Step 6: Highlighting the Current Match ```javascript function highlightCurrentMatch(index, skipScroll) { // Remove "current" styling from all highlights document.querySelectorAll(".resume-search-highlight") .forEach(function(el) { el.classList.remove("current-match"); }); if (matches[index]) { var span = matches[index].element; var highlights = span.querySelectorAll(".resume-search-highlight"); // Count how many matches in this span come before ours var matchIndexInSpan = 0; for (var i = 0; i < index; i++) { if (matches[i].element === span) matchIndexInSpan++; } var target = highlights[matchIndexInSpan]; if (target) { target.classList.add("current-match"); if (!skipScroll) { target.scrollIntoView({ behavior: "smooth", block: "center" }); } } } } ``` ### Step 7: Cleanup When the search is cleared or the user switches to a different candidate, we restore original content: ```javascript function clearHighlights() { originalContents.forEach(function(originalHTML, span) { if (span && span.parentNode) { span.innerHTML = originalHTML; } }); originalContents.clear(); } // Clear when candidate changes $scope.$watch("candidate.id", function(newVal, oldVal) { if (newVal !== oldVal) { clearSearch(); } }); // Remove event listeners on destroy $scope.$on("$destroy", function() { document.removeEventListener("keydown", handleKeydown); document.removeEventListener("keydown", handleCtrlG); clearHighlights(); }); ``` --- ## The CSS ```css .resume-search-highlight { background-color: #fff3a8; border-radius: 2px; padding: 0 1px; } .resume-search-highlight.current-match { background-color: #ff9632; box-shadow: 0 0 3px rgba(255, 150, 50, 0.5); } .resume-search-bar { position: sticky; top: 0; z-index: 100; display: flex; align-items: center; gap: 8px; padding: 6px 12px; background: #f8f9fa; border-bottom: 1px solid #e0e0e0; } ``` --- ## Edge Cases and Gotchas ### 1. Multi-page PDFs PDF.js renders each page in its own `textLayer` div. Our `querySelectorAll` naturally handles this since it selects all spans across all pages. The scroll-into-view handles cross-page navigation. ### 2. Performance on Large Resumes For a 10-page resume with thousands of text spans, searching on every keystroke can be sluggish. Solution: debounce the search by 150ms: ```javascript var searchTimeout = null; $scope.onSearchTextChange = function() { if (searchTimeout) clearTimeout(searchTimeout); searchTimeout = setTimeout(function() { performSearch($scope.searchText); }, 150); }; ``` ### 3. Special Characters in Search If the user searches for "C++", the `+` shouldn't be treated as regex. Since we use `indexOf` rather than regex, this is handled naturally. ### 4. Restoring Content After innerHTML Manipulation When we modify `span.innerHTML` to inject highlight `` elements, we lose any event listeners on those spans. Since PDF.js text layer spans are purely presentational (no interactivity), this is fine. But if your content has interactive elements, you'd need to use Range/Selection APIs instead. --- ## Takeaways 1. **`event.preventDefault()` on keydown stops native Ctrl+F** — but only if you're listening at the right level and the event hasn't already been consumed. 2. **Unicode normalization is essential** for any search that deals with real-world text. NFD decomposition + stripping combining marks gives you accent-insensitive search in 2 lines. 3. **Position mapping between normalized and original text** is the key algorithmic challenge. Walk both strings in parallel. 4. **Store original content before modifying DOM.** A Map of element -> original innerHTML makes cleanup trivial. 5. **scrollIntoView with `block: "center"`** provides the best UX for match navigation — the user sees context above and below the match. This feature shipped as two revisions: the initial Ctrl+F implementation, and a follow-up adding Ctrl+G navigation. Total code: ~320 lines of JavaScript, ~30 lines of CSS, and an HTML search bar template. --- --- title: "Preventing Architectural Decay with Custom Lint Rules" slug: abc date: 2025-08-30 updated: 2025-08-30 tags: - code description: "How a 43-line Python script saved our team from re-introducing the same structural problem every sprint." status: published --- *How a 43-line Python script saved our team from re-introducing the same structural problem every sprint.* --- ## The Backstory We had just finished extracting modules from our Django monolith. Each app now had its own API namespace, its own URL registration, its own directory. Clean boundaries. Clear ownership. Then, two weeks later, a new revision landed: ```python # In a newly created app's urls.py from auction.urls import api # <-- importing the BASE api api.register(NewResource()) # <-- registering directly into it ``` The engineer wasn't being careless — they were following the pattern they saw in older files that hadn't been migrated yet. The old pattern was the path of least resistance. This is **architectural decay**. You invest weeks decomposing a monolith, and then the structure slowly re-accumulates because the wrong pattern is still the easiest one to copy. --- ## The Insight Code review catches this *sometimes*. But code review is: - Inconsistent (depends on who reviews) - Late (the code is already written when you catch it) - Manual (doesn't scale with team size) What we needed was a machine that catches this at `arc lint` time — before the code even reaches review. --- ## The Lint Rule I wrote a custom Arcanist linter that detects two things: 1. Importing the base `api` object from `auction.urls` 2. Calling `api.register()` after such an import Here's the entire script: ```python """ Checks if the base api is imported and if so, checks if the resources are registered using the app-specific api register. If not, it leaves a lint message to fix the issue and migrate to the new API registration pattern. """ import re import sys file_to_lint = sys.argv[1] IMPORT_PATTERN = re.compile( r'from\s+auction\.urls\s+import\s+.*\bapi\b' ) REGISTER_PATTERN = re.compile(r'\bapi\.register\s*\(') with open(file_to_lint, 'r') as f: content_lines = f.readlines() has_base_api_import = False for i in range(len(content_lines)): current_line = content_lines[i] if IMPORT_PATTERN.search(current_line): has_base_api_import = True print( "%s: Do not import the base `api` from auction.urls. " "Use `get_app_api_register` instead to register APIs " "under app-specific namespace. " "See T36592 for migration guide." % (i + 1) ) if has_base_api_import and REGISTER_PATTERN.search(current_line): print( "%s: Do not register resources into the base v1 API. " "Use `register = get_app_api_register(\"\")` " "and then `register(())` instead. " "See T36592 for migration guide." % (i + 1) ) ``` 43 lines. Two regexes. That's it. --- ## How It Works in Practice When an engineer runs `arc lint` (or submits a diff for review), the linter runs on every modified Python file. If it detects the violation: ``` $ arc lint >>> Lint for urls.py: Error (check_base_api_registration) 3: Do not import the base `api` from auction.urls. Use `get_app_api_register` instead to register APIs under app-specific namespace. See T36592 for migration guide. Error (check_base_api_registration) 7: Do not register resources into the base v1 API. Use `register = get_app_api_register("")` and then `register(())` instead. See T36592 for migration guide. ``` The engineer immediately knows: - What they did wrong - What the correct pattern is - Where to find the full migration guide (the linked task) --- ## The Correct Pattern The lint message points engineers toward this approach: ```python # Before (WRONG — pollutes the global namespace) from auction.urls import api api.register(MyResource()) # After (CORRECT — app-scoped registration) from common.api_utils import get_app_api_register register = get_app_api_register("my_app") register(MyResource()) ``` The `get_app_api_register` function returns a registration helper scoped to the app's namespace. APIs registered this way appear under `/api/v1/my_app/...` instead of the flat global `/api/v1/...` namespace. --- ## Design Decisions ### Why a Script and Not an AST-Based Linter? For this specific check, regex is sufficient. The patterns we're detecting are syntactically simple: - `from auction.urls import ... api ...` - `api.register(` An AST-based approach (using `ast.parse`) would be more robust against edge cases like multi-line imports or aliased imports. But in our codebase, these patterns always appear in their simple form. The 5-minute implementation cost of regex was better than the 2-hour cost of a full AST visitor for zero practical benefit. ### Why Print Line Numbers? Arcanist's script-and-regex linter expects output in the format `LINE_NUMBER: message`. This integrates with the diff view — the lint warning appears inline next to the offending line. ### Why Check `has_base_api_import` Before `REGISTER_PATTERN`? Not every call to `.register()` is a problem. A perfectly valid pattern: ```python from my_app.api import app_api app_api.register(MyResource()) # This is fine ``` We only flag `api.register()` when `api` was specifically imported from `auction.urls`. The boolean flag creates a two-step detection: first identify the dangerous import, then flag subsequent registrations. --- ## Registering the Linter with Arcanist In `.arclint`: ```json { "linters": { "check-api-registration": { "type": "script-and-regex", "script-and-regex.script": "python scripts/lint/check_base_api_registration.py", "script-and-regex.regex": "/^(?P\\d+): (?P.*)$/m", "include": ["(\\burls\\.py$)"] } } } ``` This only runs on files named `urls.py` — where API registrations happen. Fast, targeted, no unnecessary work. --- ## Results In the 6 months after deploying this linter: - **Zero instances** of new direct base-API registration shipped to production - **3 violations caught** during development (engineers fixed them before review) - **Review bandwidth saved** — reviewers no longer had to manually check for this pattern --- ## The General Pattern: Lint as Architecture Enforcement This approach generalizes to any architectural rule that can be expressed as a code pattern: | Rule | Detection Pattern | |------|-------------------| | Don't import from base API | `from auction.urls import.*api` | | Don't use raw SQL in views | `connection.cursor()` in `views.py` | | Don't access settings directly in models | `from django.conf import settings` in `models.py` | | Don't use `print()` in production code | `\bprint\(` not in tests/ or scripts/ | | Don't import between feature modules | `from feature_a import` in `feature_b/` | The investment is tiny (30 minutes to write, 10 minutes to register). The return is permanent enforcement without ongoing human effort. --- ## Takeaways 1. **Architecture without enforcement is just a suggestion.** Decompose your monolith all you want — without lint rules, it'll re-accumulate. 2. **The error message IS the documentation.** Include the correct pattern and a link to the migration guide directly in the lint output. Don't make people search for the answer. 3. **Target your linter narrowly.** Only run on `urls.py` files. Don't scan the whole codebase on every lint run. 4. **Regex is fine for structural patterns.** You don't need a full AST parser for every lint rule. Match the tool to the problem's complexity. 5. **The best lint rules prevent problems that code review catches inconsistently.** If reviewers sometimes miss it, automate it. 6. **43 lines of prevention > 43 hours of cleanup.** The cost-benefit ratio of simple automated checks is extraordinarily high. --- --- title: "Adding Structured Logging to Celery Tasks Without Touching Every Task" slug: structured-logging-celery-tasks date: 2025-08-08 updated: 2025-08-08 tags: - code description: "How a single signal handler gave us argument-level observability across 50+ background jobs." status: published --- *How a single signal handler gave us argument-level observability across 50+ background jobs.* --- ## The Problem We had 50+ Celery tasks across our Django application — sending emails, syncing data, generating reports, processing uploads. When something went wrong, the debugging flow looked like this: 1. Alert fires: "Task `send_candidate_email` failed" 2. Open Sentry → see the traceback 3. Traceback shows a `KeyError` or `TypeError` 4. But **what arguments were passed?** Who was this for? Which candidate? Which job? 5. Grep through application logs → nothing useful 6. Try to reproduce manually → waste 30 minutes guessing inputs The issue: our tasks logged their *existence* (start/success/failure) but not their *arguments*. Without knowing what data was passed, debugging was guesswork. --- ## The Naive Solution (Don't Do This) The obvious approach: add logging to every task. ```python @celery_task def send_candidate_email(candidate_id, template_name, **kwargs): logger.info(f"send_candidate_email called with " f"candidate_id={candidate_id}, " f"template_name={template_name}, " f"kwargs={kwargs}") # ... actual task logic ``` This has several problems: 1. **You have to modify every task.** 50+ tasks means 50+ changes. 2. **New tasks forget to add it.** No enforcement mechanism. 3. **Inconsistent formatting.** Each developer logs differently. 4. **Verbose and noisy.** The logging line is often longer than the task logic. --- ## The Better Solution: Celery Signals Celery provides signals that fire at various points in a task's lifecycle. The one we want is `task_prerun` — fires after the task is received by a worker but before execution begins. ```python from celery.signals import task_prerun @task_prerun.connect def on_task_prerun(sender, task_id, args, kwargs, **extra): logger.info( 'Task %s[%s] | args=%r | kwargs=%r', sender.name, task_id, args, kwargs ) ``` That's it. Four lines. Every task in your application now logs its arguments before execution. --- ## Where to Put It This goes in your Celery app configuration — the file where you define your Celery instance: ```python # project/celery.py import logging from celery import Celery from celery.signals import task_prerun logger = logging.getLogger('celery.tasks') app = Celery('myproject') app.config_from_object('django.conf:settings', namespace='CELERY') app.autodiscover_tasks() @task_prerun.connect def on_task_prerun(sender, task_id, args, kwargs, **extra): logger.info( 'Task %s[%s] | args=%r | kwargs=%r', sender.name, task_id, args, kwargs ) ``` Because this is registered at the Celery app level via signals, it applies to every task that runs on this worker — including tasks in third-party packages. --- ## What the Output Looks Like ``` [INFO] celery.tasks: Task send_candidate_email[a3f2b1c4-...] | args=(42851, 'interview_confirmation') | kwargs={'cc': ['hr@company.com']} [INFO] celery.tasks: Task sync_calendar_events[7e8d9f01-...] | args=() | kwargs={'recruiter_id': 1523, 'force': True} [INFO] celery.tasks: Task generate_report[b4c5d6e7-...] | args=(891,) | kwargs={'format': 'xlsx', 'date_range': '2024-01-01:2024-12-31'} ``` Now when a task fails, the arguments are right there in the logs — same timestamp, same request. No guessing. --- ## Signal Parameters Explained ```python @task_prerun.connect def on_task_prerun(sender, task_id, args, kwargs, **extra): # sender: The task class (has .name attribute) # task_id: UUID for this specific execution # args: Positional arguments tuple # kwargs: Keyword arguments dict # extra: Additional signal metadata (usually empty) ``` ### Other Useful Signals | Signal | Fires When | Use Case | | --------------- | ------------------------- | ------------------- | | `task_prerun` | Before task executes | Log arguments | | `task_postrun` | After task completes | Log result/duration | | `task_success` | On successful completion | Metrics/counters | | `task_failure` | On exception | Alert with context | | `task_received` | When worker receives task | Queue monitoring | You can compose these for richer observability: ```python from celery.signals import task_prerun, task_postrun, task_failure import time _task_start_times = {} @task_prerun.connect def on_task_prerun(sender, task_id, args, kwargs, **extra): _task_start_times[task_id] = time.time() logger.info( 'Task %s[%s] started | args=%r | kwargs=%r', sender.name, task_id, args, kwargs ) @task_postrun.connect def on_task_postrun(sender, task_id, retval, state, **extra): start = _task_start_times.pop(task_id, None) duration = f"{time.time() - start:.2f}s" if start else "unknown" logger.info( 'Task %s[%s] completed | state=%s | duration=%s', sender.name, task_id, state, duration ) @task_failure.connect def on_task_failure(sender, task_id, exception, args, kwargs, **extra): logger.error( 'Task %s[%s] FAILED | exception=%s | args=%r | kwargs=%r', sender.name, task_id, repr(exception), args, kwargs ) ``` --- ## Handling Sensitive Arguments Not all arguments should be logged. You might pass passwords, tokens, or PII: ```python SENSITIVE_TASKS = { 'auth.reset_password', 'payments.process_charge', } REDACT_KWARGS = {'password', 'token', 'secret', 'credit_card'} @task_prerun.connect def on_task_prerun(sender, task_id, args, kwargs, **extra): if sender.name in SENSITIVE_TASKS: logger.info('Task %s[%s] started (args redacted)', sender.name, task_id) return safe_kwargs = { k: '***' if k in REDACT_KWARGS else v for k, v in kwargs.items() } logger.info( 'Task %s[%s] | args=%r | kwargs=%r', sender.name, task_id, args, safe_kwargs ) ``` --- ## Handling Large Arguments Some tasks receive large payloads (file contents, bulk data). You don't want megabytes in your logs: ```python import sys MAX_ARG_LOG_SIZE = 1024 # bytes def truncate_repr(obj, max_size=MAX_ARG_LOG_SIZE): r = repr(obj) if len(r) > max_size: return r[:max_size] + f'... (truncated, {sys.getsizeof(obj)} bytes)' return r @task_prerun.connect def on_task_prerun(sender, task_id, args, kwargs, **extra): logger.info( 'Task %s[%s] | args=%s | kwargs=%s', sender.name, task_id, truncate_repr(args), truncate_repr(kwargs) ) ``` --- ## Integration with APM Tools If you use Datadog, New Relic, or similar APM tools, you can enrich the trace span: ```python from ddtrace import tracer @task_prerun.connect def on_task_prerun(sender, task_id, args, kwargs, **extra): span = tracer.current_span() if span: span.set_tag('celery.task_name', sender.name) span.set_tag('celery.task_id', task_id) # Add searchable tags for key arguments if args and isinstance(args[0], int): span.set_tag('celery.primary_id', args[0]) ``` Now you can search in Datadog: "Show me all executions of `send_email` where `primary_id = 42851`." --- ## Testing the Signal ```python # tests/test_celery_signals.py from unittest.mock import patch from myapp.tasks import send_candidate_email @patch('project.celery.logger') def test_task_prerun_logs_arguments(mock_logger): send_candidate_email.apply(args=(123, 'welcome'), kwargs={'cc': ['a@b.com']}) mock_logger.info.assert_called() call_args = mock_logger.info.call_args[0] assert 'send_candidate_email' in call_args[0] or call_args[1] assert '123' in str(call_args) ``` --- ## Results After deploying this single signal handler: - **Mean time to debug task failures:** Reduced from ~30 minutes to ~5 minutes - **"Cannot reproduce" bugs:** Dropped significantly — we now have the exact inputs - **Zero code changes to existing tasks:** The signal applies globally - **Automatic coverage of new tasks:** Any new task immediately gets logging --- ## Takeaways 1. **Celery signals are the right abstraction for cross-cutting concerns.** Don't modify individual tasks when you can hook into the lifecycle. 2. **`%r` (repr) is better than `%s` (str) for logging arguments.** It shows types, distinguishes `None` from `"None"`, and handles nested structures. 3. **Log at the `INFO` level, not `DEBUG`.** You want this in production. `DEBUG` often gets filtered out. 4. **Handle sensitive data.** Add a redaction layer from day one. Logging passwords to your aggregation service is a security incident. 5. **This pattern applies beyond Celery.** Django signals, middleware, decorators — any framework that provides lifecycle hooks can be instrumented this way without modifying business logic. The total investment: 4 lines of code in one file. The total return: permanent argument-level observability for every background job in the system. --- --- title: "Decomposing A Django Monolith" slug: django-monolith-decomposition date: 2025-07-02 updated: 2025-07-02 tags: - code description: "How I extracted two full-stack modules from a production Django app without downtime — and what I'd do differently next time." status: published --- *How I extracted two full-stack modules from a production Django app without downtime — and what I'd do differently next time.* --- ## The Problem Our product had grown over several years into a classic Django monolith. One top-level app (`auction`) contained everything — models, views, APIs, templates, JS controllers, CSS files — all tangled together. A single change to the evaluations feature could break the resume viewer. CSS specificity wars were constant. New engineers took weeks to understand which files belonged to which feature. The codebase looked something like this: ``` auction/ ├── models.py # 3000+ lines, 40+ models ├── views.py # 2000+ lines ├── api.py # All Tastypie resources in one file ├── forms.py # Every form for every feature ├── static/ │ ├── js/ # 100+ controllers, all in one folder │ └── css/ # One massive main.less importing everything └── templates/ └── auction/ # 200+ templates, flat hierarchy ``` The task: extract the **Evaluations** module and the **Resume Modal** module into independent Django apps, each owning their own models, APIs, templates, JS, and CSS. --- ## Why This Is Hard Monolith decomposition sounds straightforward — "just move the files" — but in practice you hit a wall of invisible dependencies: 1. **Circular imports.** Model A references Model B which imports a helper from Model A's module. 2. **Template inheritance.** Templates extend base layouts that assume certain CSS/JS is globally available. 3. **Shared state in JS.** AngularJS controllers share scope variables across features. 4. **Foreign key migrations.** Django migrations reference the old app label. Moving a model means rewriting migration history. 5. **CSS specificity.** Moving a LESS file breaks the import order, changing which styles win. 6. **VCS history.** A naive copy-delete loses all git/hg blame history. --- ## The Methodology I developed a phased approach (which I later documented internally for the team). Here's the process: ### Phase 1: Dependency Mapping Before touching any code, I mapped every dependency of the target module: ```python # What I needed to answer for each file: # 1. What does this file import FROM other modules? # 2. What do other modules import FROM this file? # 3. What templates does this include/extend? # 4. What JS services does this controller depend on? ``` I used `grep` extensively: ```bash # Find all imports of evaluation-related symbols rg "from.*models import.*Evaluation" --type py rg "evaluationService|evaluationCtrl" --type js rg "evaluation" templates/ --include="*.html" ``` This produced a dependency graph. The key insight: **you need to identify the cut points** — the minimal set of interfaces between your module and the rest of the system. ### Phase 2: Create the Target App Structure ```bash # Create the new app with Django's startapp python manage.py startapp evaluations auction/resume_modal/evaluations # But we need more than just Python: mkdir -p auction/resume_modal/evaluations/{static,html,migrations} mkdir -p auction/resume_modal/evaluations/static/evaluations/{js,css} mkdir -p auction/resume_modal/evaluations/html/evaluations ``` ### Phase 3: Move Python Code (Preserve History) This is critical. Don't copy-paste. Use your VCS's move command: ```bash # Mercurial hg mv auction/models/evaluation.py auction/resume_modal/evaluations/models.py # Git equivalent git mv src/models/evaluation.py src/evaluations/models.py ``` This preserves blame history. Reviewers can still see who wrote each line and why. **The models after extraction:** ```python # auction/resume_modal/evaluations/models.py from django.db import models from django.db.models import Q, Count, Avg class EvaluationManagerMixin(object): """ Parent class for both the evaluation queryset and manager. """ def past_evaluations(self, opp, rec_ids, inner_facet, job_stage_round): if not inner_facet: inner_facet = opp.get_inner_facet() stage_filter = Q(inner_facet=inner_facet) if inner_facet >= common.choices.INNER_FACETS["CUSTOM_STAGES"]: stage_filter = Q(stage__position=inner_facet) return self.filter( stage_filter, recruiter__id__in=rec_ids, opportunity=opp, job_stage_round_id=job_stage_round ) def pending_evaluations(self): return self.filter(completed_at__isnull=True) class EvaluationQuerySet(EvaluationManagerMixin, models.QuerySet): pass class EvaluationManager(EvaluationManagerMixin, models.Manager): def get_queryset(self): return EvaluationQuerySet(self.model, using=self._db) ``` ### Phase 4: Move JavaScript (Controllers + Services) AngularJS modules had to be moved carefully because the controller registration order matters: ```javascript // Before: everything registered on one global module angular.module('app') .controller('EvaluationCtrl', function($scope, evaluationService) { // 500 lines of evaluation logic }); // After: controller lives in its own file under the evaluations app // auction/resume_modal/evaluations/static/evaluations/js/evaluationCtrl.js angular.module('app') .controller('EvaluationCtrl', ['$scope', 'evaluationService', function($scope, evaluationService) { // Same logic, but now in the correct directory } ]); ``` The key: update all `