Reference private file?

I'm making migration plugin which I'm using in first migration and it is creating private file like this:

if ($final_destination) {
  // Create a file entity.
  $file = File::create([
    'uri' => $final_destination,
    'uid' => $uid,
    'status' => FILE_STATUS_PERMANENT,
  ]);

  $uri = $file->getFileUri();

  $stream = new S3fsStream();
  $stream->writeUriToCache($uri);

  if ($stream->waitUntilFileExists($uri)) {
    $file->save();
    return $file->getFileUri();
  }
}

Then, second migration should create media entities, by using those files (with migration lookup plugin).

However, when I run second migration I get error message:

media: field_image.0=You do not have access to the referenced entity file: 46

If file is public then it works, but how can I reference private files? The error is triggered in ReferenceAccessConstraintValidator.php.

  // We check that the current user had access to view any newly added
  // referenced entity.
  if ($check_permission && !$referenced_entity->access('view')) {
    $type = $value->getFieldDefinition()->getSetting('target_type');
    $this->context->addViolation($constraint->message, ['%type' => $type, '%id' => $id]);
  }