n8n
A market feed in an n8n loop, paginated by following one URL, and what to do about the daily budget.
One HTTP Request node reads a whole market. The feed's next_url is absolute and complete, so n8n's built-in pagination handles everything.
The HTTP Request node
| Setting | Value |
|---|---|
| Method | GET |
| URL | https://prospex.ch/app/markets/feed/<token>.json?limit=200 |
| Authentication | None. The token is in the path. |
| Pagination mode | Response Contains Next URL |
| Next URL | {{ $response.body.next_url }} |
| Complete when | {{ !$response.body.next_url }} |
Put the URL in a credential or an environment variable rather than in the node.
Getting to one item per company
Each page arrives as one item holding the envelope. Add a Split Out node on the data field and every company becomes its own item, with the flat field names available to every node downstream:
{{ $json.company_name }}
{{ $json.company_domain }}
{{ $json.fit }}
{{ $json.reasons.join(", ") }}
Three fields are arrays: reasons, missing and signals. Join the first two for a message, and split out signals again if you want one item per event.
Handling a refusal
Turn off "Continue On Fail" and let the node stop, then read the body: every refusal answers JSON with a stable error value.
daily_row_limit: you have taken the day's 5,000 rows. Readretry_afterfrom the body, or schedule the workflow later.invalid_parameter: the market rebuilt mid-run and your cursor belongs to the old order. Start the run again from the first page.unknown_token: the URL was rotated. Copy the new one from the Connect page.subscription_required: billing. The body carries the URL to fix it.
A run that walks a whole market spends as many rows as the market has companies. Schedule it on the cadence your market rebuilds on rather than hourly, and check the usage block on each page. Limits has the rest.
Driving a Prospect project instead
The same node shape works against the API, with two additions: an Authorization: Bearer header, and an Idempotency-Key on every mutation. Use the execution id for the key. Send a CRM segment into a project is that workflow written out.