← Back to Explorer

About & Methodology

How this tool works, what we discovered building it, and why it matters.

1. Why This Tool Exists

NYC's 76 Business Improvement Districts (BIDs) manage streetscape, sanitation, and public safety in commercial corridors across all five boroughs. To understand quality-of-life issues in their district, BID managers need to see 311 complaint patterns — what types of complaints are filed, where they cluster, how quickly they're resolved, and how trends change over time.

The existing NYC 311 portal at portal.311.nyc.gov/check-status/ offers filters for borough, address radius, problem type, and date range — but not by BID. There is no way to draw a BID boundary and see all complaints within it. This tool fills that gap.

Existing Portal Filter Parameters

Parameter	Type	Limitation
borough	Dropdown	Too broad — a borough contains dozens of BIDs
minlatitude / maxlatitude / minlongitude / maxlongitude	Bounding box	Rectangular only, can't follow BID boundaries
fromdate / todate	Date range	Works well, no issues
problemarea / problem	CRM option set IDs	Internal Dynamics 365 values, not documented
communitydistrict	Text	BIDs don't align with community districts

None of these parameters support polygon-based spatial queries. A BID manager would need to manually adjust a bounding box, visually filter results, and repeat for each date range — an impractical workflow for ongoing monitoring.

2. Data Sources

NYC Open Data — BID Boundaries (Geography Only)

NYC Open Data supplies the parcel polygons used to define each Business Improvement District. It is used for geography only; the 311 requests displayed by this tool come from the live Portal.

• Endpoint: data.cityofnewyork.us/resource/7jdm-inj8.geojson
• CORS: Fully open (Access-Control-Allow-Origin: *)
• Contents: BID names, boroughs, websites, and tax-lot parcel polygons

NYC 311 Portal — Internal API

The 311 portal at portal.311.nyc.gov runs on Microsoft Dynamics 365 Power Apps Portals. Its internal map endpoint is the tool's only source of 311 request records.

• Endpoint: portal.311.nyc.gov/entity-pin-fetch-service-requests/
• CORS: Locked to www1.nyc.gov only
• Record limit: 100 records maximum, no pagination
• Response: Returns HTML page wrapping JSON data
• Dates: The server normalizes the feed's unzoned UTC timestamps and displays them in New York time

Available 311 Portal Fields

Field	Map Feed	Case Detail Page
Service-request number	Available	Available
Request type and status	Available	Available
Address and coordinates	Available	Address available
Reported date	Available	Available
Problem and additional details	Not included	Available when provided
Portal update and closed dates	Not included	Available when provided
Expected next update	Not included	Available when provided
Agency, channel, and resolution	Not included	Not reliably available

3. How We Discovered the Portal API

The 311 portal's internal API is not documented. We discovered it by reverse-engineering the JavaScript on the /check-status/ page. The portal uses an Esri ArcGIS map with clustered markers, and a FilterManager class that aggregates multiple filter types before sending requests to the backend.

The key endpoint is /entity-pin-fetch-service-requests/, which accepts GET parameters for borough, date range, bounding box coordinates, and CRM-internal option set IDs for problem area and problem type. Filter option values are loaded dynamically from:

• /get-optionset-filter-options/?entity=n311_srproblem&attribute=n311_problemareaglobal
• /get-lookup-filter-options/?lookupentity=n311_srproblem&lookupidattribute=n311_srproblemid&...

Each pin in the response contains:

{
  "id": "70f3d629-59e0-ef11-95f5-7c1e52e4043a",  // Dynamics 365 CRM GUID
  "label": "Noise - Street/Sidewalk",
  "sublabel": "616 WEST 184 STREET, MANHATTAN (NEW YORK), NY, 10033",
  "latitude": "40.8513685798",
  "longitude": "-73.9334866503",
  "data": {
    "address": "616 WEST 184 STREET, MANHATTAN (NEW YORK), NY, 10033",
    "problem": "Noise - Street/Sidewalk",
    "submitteddate": "2/1/2025 4:59:48 AM",
    "status": "Closed",
    "srnumber": "311-21789902"
  }
}

Portal Limitations

How We Bypass the 100-Record Cap

A single portal request for a BID over a 30-day range might contain 500+ complaints, but the API only returns the 100 most recent. Pagination parameters don't exist. To capture all records, our server uses an adaptive multi-phase strategy:

Phase 1: Daily Time Windows

Instead of one request for the full date range, the server splits it into one request per day. Most BIDs receive 20-50 complaints per day — well under the 100 cap. For a 30-day query, this means 30 parallel requests (running 8 at a time) instead of 1, capturing nearly 100% of records.

Phase 2: Spatial Subdivision (Fallback)

If any single day returns exactly 100 records (meaning it hit the cap), that day is automatically re-fetched using spatial tiling. The BID's bounding box is divided into a 2×2 grid of smaller sub-boxes, and each tile is queried separately for that day. This works because the portal's date filter only accepts whole dates — you can't split by time-of-day, so space is the only remaining axis to subdivide.

In practice, Phase 2 rarely triggers. Even Times Square — one of NYC's busiest BIDs — stayed under 100 complaints per day across a full month of testing. Phase 2 exists as a safety net for extreme cases (e.g., a major event or outage generating a surge of complaints in one district on one day).

Interactive Demo: How Spatial Splitting Works

Use the slider below to simulate increasing complaint volume for a single day in the Hudson Square BID (bounding box: 819m × 530m). Watch what happens when the 100-record portal cap is reached.

Phase 3: Deduplication & Caching

After spatial tiling, records near tile boundaries may appear in multiple tiles. The server deduplicates by SR number before returning results. The complete result set is then cached in memory for 10 minutes, so repeat queries are instant.

Phase	What It Does	When It Runs	Calls
1	Split date range into daily windows, fetch in parallel (8 concurrent)	Always	1 per day
2	Re-split capped days into 2×2 spatial tiles	Only if a day hits 100	4 per capped day
3	Deduplicate by SR number + cache 10 min	Always	0

Scenario	Total Calls	Estimated Time
Quiet BID, 7 days	7	<1 second
Busy BID, 30 days (no caps)	30	~3 seconds
Busy BID, 30 days (2 capped days)	30 + 8 = 38	~4 seconds

4. Portal-Only 311 Data

• Near-real-time results from the same source residents use to check requests
• Portal service-request numbers and direct links to request detail pages
• Portal-native statuses, request types, reported dates, addresses, and coordinates
• One source of truth for map pins, counts, filters, and summaries

The Portal limits each response to 100 records and does not paginate. The server therefore queries one day at a time and subdivides capped days into smaller map areas. When a case popup opens, the server fetches that case's Portal detail page for problem details, update timing, closure timing, and the expected next update when those values are available. The tool does not infer unavailable agency, channel, or resolution fields from another dataset.

5. Official BID Boundaries

The BID boundary dataset from NYC Open Data (data.cityofnewyork.us/resource/7jdm-inj8) does not contain district boundaries. Instead, it contains tax lot parcel polygons — the individual building footprints within each BID. Streets, sidewalks, and public rights-of-way are excluded.

The Problem

BID	Polygon Rings	Area Covered	Street Gaps
Times Square Alliance	45	36%	64%
Alliance for Downtown NY	119	37%	63%
Village Alliance	40	12%	88%
Union Square Partnership	20	13%	87%
Diamond District	3	31%	69%

If we used the raw polygons for point-in-polygon filtering, we would miss 60-88% of 311 requests — especially street-level complaints (noise, potholes, sidewalk conditions, encampments) which are geocoded to street centerlines.

The Solution: Two Complementary Sources

The application uses NYC's Business Improvement District ArcGIS feature layer as the authoritative district boundary. These continuous polygons include streets inside each BID and follow the official outer edge. The Open Data parcel dataset remains useful for parcel detail, district websites, founding years, and display names.

Source	Geometry	Purpose
NYC ArcGIS	Official district polygons	Selection, request filtering, map framing, and the yellow boundary
NYC Open Data	Tax lot parcel polygons	Blue parcel detail and BID descriptive metadata
Fallback	Gap-closed parcel geometry	Used only if the ArcGIS boundary service is temporarily unavailable

On the map, you can see both layers: the blue dashed outlines are the raw tax lot parcels, and the yellow solid outline is the official district boundary used for filtering. No artificial exterior buffer is added.

6. Coordinate Handling

Map pins use the coordinates returned by the 311 Portal. After the server fetches pins within the selected BID's bounding box, the browser applies a point-in-polygon test against the official BID boundary. This removes nearby requests that fall inside the rectangular query area but outside the district. If two boundaries genuinely overlap, a point is assigned only to the BID whose original tax parcels are closest, preventing cross-BID double counting.

7. How the Map Works

Data Flow

1. User selects a BID from the dropdown
2. The corresponding official ArcGIS district polygon is selected
3. A bounding box (with 50m query padding) is computed around the official boundary
4. The server proxy queries the 311 Portal by date and bounding box, splitting the range into daily windows
5. Days that reach the Portal's 100-record cap are queried again in a 2x2 spatial grid
6. Returned points are filtered client-side against the official boundary and assigned to the nearest original BID parcels if boundaries overlap
7. Pins are plotted and color-coded by complaint type

Available Portal Fields

Portal Map Feed	Portal Case Detail	Not Reliably Available
Request type, status, and reported date	Problem and additional details	Agency and submission channel
Address and coordinates	Portal update and closure timing	Resolution narrative
Service-request number and detail link	Expected next update	Structured BBL and community board

8. Future Expansion

This tool's architecture — BID polygons + geospatial data overlay — can be extended to incorporate other NYC datasets for a comprehensive district dashboard:

9. Technical Stack

Component	Technology	Why
Server	Node.js + Express	Proxies portal API (CORS bypass), serves static files, easy deployment
Map	Leaflet + MarkerCluster	Free, lightweight, handles thousands of pins with clustering
Spatial Processing	Turf.js	Spatial source matching, point-in-polygon filtering, and bounding box computation
Charts	Chart.js	Lightweight charting with no build step
Date Picker	Flatpickr	Tiny, range-mode support, dark theme
Heatmap	Leaflet.heat	Density visualization layer
311 Request API	NYC 311 Portal (proxied)	Live request pins, statuses, and detail links
Boundary APIs	NYC ArcGIS + NYC Open Data	Official BID boundaries plus parcel geometry and district metadata

311 requests: live NYC 311 Portal only · BID geography: NYC ArcGIS and NYC Open Data