PimPagePrefetcher
in package
Walks a paginated PIM endpoint in order while keeping several page requests in flight.
A products populate spent ~80% of its ten minutes waiting on PIM, one 50-item page at a time (5-6 s each). PIM serves pages concurrently — four in parallel came back in half the time of four in sequence — so the cost was our serial loop, not the API. The consumer (populateCollection()) still asks for page N, then N+1; this sends the following pages ahead, so PIM renders them while we transform and import the current one.
The first page is fetched alone: its count says how many pages exist, so nothing is ever
requested past the end. That also removes the trailing 404 every run used to log.
Table of Contents
Constants
- DEFAULT_CONCURRENCY : mixed = 6
Properties
- $concurrency : int
- $fetch : callable(string, int, mixed): PromiseInterface
- $inFlight : array<int, PromiseInterface>
- $lang : mixed
- $lastPage : int|null
- $nextToRequest : int
- $ready : array<int, array<string|int, mixed>>
- $requestUrl : string
Methods
- __construct() : mixed
- page() : array<string|int, mixed>
- Returns one page in the shape of PimService::fetchSinglePage().
- endOfPages() : array<string|int, mixed>
- learnLastPage() : void
- request() : void
- settle() : void
- topUp() : void
Constants
DEFAULT_CONCURRENCY
public
mixed
DEFAULT_CONCURRENCY
= 6
Properties
$concurrency
private
int
$concurrency
$fetch
private
callable(string, int, mixed): PromiseInterface
$fetch
$inFlight
private
array<int, PromiseInterface>
$inFlight
= []
Pages requested and still in flight.
$lang
private
mixed
$lang
= null
$lastPage
private
int|null
$lastPage
= null
$nextToRequest
private
int
$nextToRequest
= 1
$ready
private
array<int, array<string|int, mixed>>
$ready
= []
Pages fetched and not yet handed out.
$requestUrl
private
string
$requestUrl
Methods
__construct()
public
__construct(string $requestUrl[, mixed $lang = null ][, int|null $concurrency = null ][, callable|null $fetch = null ]) : mixed
Parameters
- $requestUrl : string
- $lang : mixed = null
- $concurrency : int|null = null
- $fetch : callable|null = null
-
Page fetcher with PimService::fetchPageAsync()'s signature; a test seam.
page()
Returns one page in the shape of PimService::fetchSinglePage().
public
page(int $page) : array<string|int, mixed>
Pages are expected in ascending order. Past the last page an empty 200 comes back, which populateCollection() reads as "done" without logging anything.
Parameters
- $page : int
Return values
array<string|int, mixed>endOfPages()
private
static endOfPages() : array<string|int, mixed>
Return values
array<string|int, mixed>learnLastPage()
private
learnLastPage(int $page, array<string|int, mixed> $result) : void
Parameters
- $page : int
- $result : array<string|int, mixed>
request()
private
request(int $page) : void
Parameters
- $page : int
settle()
private
settle(int $page) : void
Parameters
- $page : int
topUp()
private
topUp() : void