Search

Keeping the keys fresh

Search keys are computed at indexing time. Declare with reindexWhen() and ReindexesSearch what reindexes the documents when the data behind them changes.

The keys of a document are computed when it is indexed. Scout indexes a document again when its own model is saved. When a key reads another model, such as the client of the project of a post, a change of that model leaves the documents with their former keys: a former key can show a document to a reader who lost access.

reindexWhen()

Declare what reindexes the documents on the perimeter whose keys read the other model, after searchKeys():

ClientPerimeter::new()
    ->allowed(fn (User $user, Ability $ability) => $user->client_id !== null && $user->can("{$ability} client posts"))
    ->query(fn (Builder $query, User $user) => $query->whereRelation('project', 'client_id', $user->client_id))
    ->searchKeys(
        name: 'client',
        row: fn (Post $post) => $post->project->client_id,
        reader: fn (User $user) => $user->client_id,
        load: ['project:id,client_id'],
    )
    ->reindexWhen(Project::class, changed: ['client_id'], rows: fn (Project $project) => $project->posts()),

When a project changes client, a job reindexes its posts, in chunks, after the commit.

  • The rows are reindexed when a Project is updated and one of the changed attributes changed, as Eloquent's wasChanged() compares them, and when a Project is created, deleted, soft deleted, force deleted or restored. A save that changes no listed attribute queues nothing.
  • rows receives the project and returns a query or a relation of the controlled model. The rows of the project's previous state are reindexed too, which matters when the watched model holds the link, such as an assignment moved from one post to another.
  • A perimeter may watch several models, with one reindexWhen() each.

The job

  • It is queued after the commit, always, whatever Scout's after_commit option, on Scout's queue connection and queue (scout.queue). Without a queue, it runs synchronously after the commit. A rolled back transaction queues nothing.
  • It reindexes the rows in chunks of scout.chunk.searchable, through Scout, and removes from the index the rows whose shouldBeSearchable() is now false.
  • It carries no full copy of the watched model: its key, and for an update the previous values of the changed attributes. For a deleted model, it carries its attributes without the $hidden columns, so a rows callback must not read a hidden attribute.
  • Inside withoutSyncingToSearch() of the controlled model, nothing is queued.

Discovery

A queue worker that only updates a project must know that posts depend on it, without building PostControl. The package listens to the created, updated, deleted and restored events of every model and looks up the controls that watch it: about a microsecond for a model that nobody watches.

  • A control that declares reindexWhen() must be discovered, in app/Access/Controls or a directory passed to Access::discoverControlsIn(). A control linked only by #[ControlledBy] throws UnwatchableReindexException when it is built.
  • So does a control whose model does not use ControlledSearchable, unless a searchable child inherits its keys: see Searching through a parent.
  • php artisan access:cache caches the watched models with the controls, and the cache is used as it is: run it in every deployment, which php artisan optimize does. A stale cache throws UnwatchableReindexException when a control whose reindexWhen() it does not know is built.

Errors

MistakeException
reindexWhen() before searchKeys() on the perimeter, or without itReindexWithoutSearchKeysException
A class that is not an Eloquent modelInvalidReindexModelException
An empty or blank attribute in changedInvalidReindexAttributesException
A rows callback that does not accept the model, or returns anything but a query or relation of the controlled modelInvalidReindexRowsException
The same model watched twice, or a parent or child class of a watched modelDuplicateReindexException

Pivots

A key that reads a many-to-many relation, such as the users a post is shared with, is kept fresh by a custom pivot that uses the ReindexesSearch trait:

use Illuminate\Database\Eloquent\Relations\Pivot;
use Lomkit\Access\Search\ReindexesSearch;

class PostShare extends Pivot
{
    use ReindexesSearch;
}
app/Models/Post.php
public function sharedWith(): BelongsToMany
{
    return $this->belongsToMany(User::class)->using(PostShare::class);
}
app/Models/User.php
public function sharedPosts(): BelongsToMany
{
    return $this->belongsToMany(Post::class)->using(PostShare::class);
}
  • attach(), detach(), sync(), toggle() and updateExistingPivot() on a relation that uses the pivot fire its events, and each event reindexes the linked models that use ControlledSearchable.
  • Declare ->using(PostShare::class) on both sides of the relation: otherwise attach() and detach() from the other side write the pivot table with a plain query, fire no event, and the documents keep stale keys. access:check reports a relation that reaches the table without it (pivot_without_using).
  • The job is unique per document and queued after the commit: fifty attaches queue one job per post. Without a queue, the jobs run as the pivots are written, so wrap a bulk change in DB::transaction() to reindex each document once.
  • A pivot written outside its relation, such as PostShare::create([...]), does not know the models it links: the write throws UnresolvablePivotException before it reaches the database. A pivot that links no ControlledSearchable model throws MissingControlledSearchableException.
  • The pivot keys must reference the primary keys of the models: a relation with a custom parentKey or relatedKey is not supported.

What Eloquent cannot see

A write without Eloquent events reindexes nothing. Reindex the rows yourself, through Scout:

Post::where('project_id', $id)->update(['author_id' => $newAuthor]); // a mass update fires no event
Post::where('project_id', $id)->searchable();                       // Scout reindexes them in chunks

$post->sharedWith()->attach($user); // a plain pivot, without ReindexesSearch
$post->searchable();

The same goes for DB::table() writes, Model::withoutEvents(), withoutSyncingToSearch(), pivot queries such as newPivotStatement(), and cascades or SET NULL done by the database.

The document's own model

The model of the document needs nothing: Scout reindexes it when it is saved. When the model overrides searchIndexShouldBeUpdated() and it returns false, the package compares the keys of the row before and after the save, after the commit, each state with its own load relations, and reindexes the document when they differ. Keep your override for the other attributes.

Turn on Scout's after_commit option, so that Scout's own indexing never writes the keys of a change that is rolled back:

config/scout.php
'after_commit' => true,

After a deployment

Reindex each time the row side of a key or the boundary changes, as after a change of toSearchableArray():

php artisan scout:sync-index-settings
php artisan scout:import "App\Models\Post"
  • scout:import updates the documents in place, and until it has run they keep the keys of the former rule. When you narrow a rule to close an access, run it right away; to close it at once, run scout:flush before scout:import.
  • With Typesense, delete the collection first (php artisan scout:delete-index posts) when its schema changes, so that it is created again with the new fields.
  • A change of the reader side needs no reindex: the keys of a reader are computed at each search.

The window

Freshness has a limit: between a change and the end of its job, the documents keep their former keys. Without a queue, the window closes right after the commit. With one, it lasts as long as the backlog of the queue, and a failed job leaves the keys stale until it is retried or the rows are reindexed.