Issue #3605812: Add a setup-provider agent skill to configure the OpenAI provider from an environment variable

Joshua Fernandesrequested to merge
issue/ai_provider_openai-3605812:3605812-create-a-skill into 1.3.x

Closes #3605812.

Adds .agents/skills/setup-provider/SKILL.md — an agent skill that sets up the OpenAI provider on a running site from the CLI, reading the API key from an existing environment variable and never touching the secret itself.

Follows the pattern the two AI-core skills settled on (ai#3586561, ai#3586562): a single SKILL.md, driven by drush key:save plus core config commands. No tool plugins, no drupal/tool dependency, no PHP added — so there is nothing here to unit test and no new dependency to carry.

What the skill does

  1. Blocking clarification step — asks for the environment variable name only, and for the Key entity ID/label.
  2. Guidance for obtaining a key at platform.openai.com/api-keys, and for setting it in the environment for ddev, PHP-FPM, and Docker/Kubernetes.
  3. Confirms the variable is set without reading its value — ${#VAR} for the character count, and printenv | cut -d= -f1 | grep -cx (names only) to distinguish a persistent variable from one prefixed onto a single command.
  4. Creates a Key entity with the Key module's env provider via drush key:save, so the value is read at request time and never enters config, the database, or a config export.
  5. Writes the key's ID to ai_provider_openai.settings:api_key.
  6. Seeds ai.settings:default_providers from OpenAiProvider::getSetupData().
  7. Optional host and moderation, then an end-to-end connectivity check and a report, plus a rollback section.

Secret handling

The acceptance criterion is that the value is never exposed or logged, so the skill is explicit about it: existence and length are the only facts it reads. drush key:value-get is banned outright — it prints the value in a table and --format does not exist on that command under Drush 13. The env provider also rejects a key value passed on the command line, which removes the possibility of the secret landing in shell history or the process list.

Failure modes documented (all verified on a live site)

The skill's value is mostly in these, since each one saves cleanly and fails later somewhere else:

  • PHP-FPM vs Drush environment. A variable prefixed onto a command resolved correctly under Drush while a web request to the same site saw nothing; container-level variables were visible to both. clear_env defaults to yes in PHP-FPM and wipes the inherited environment; ddev ships clear_env = no. A check that passes in your shell proves nothing.
  • key:save does not validate that the variable exists — the env provider only checks that in its configuration form, so a key pointing at a nonexistent variable saves with no error and resolves to NULL forever.
  • The error stops naming the cause. With api_key pointing at an unresolvable key, getConfiguredModels() logs "Could not load the OpenAI API key" and then issues the request anyway, so what surfaces is OpenAI\Exceptions\ErrorException: Missing bearer authentication in header. Hence verify-then-wire-up ordering.
  • drush config:set … moderation false stores TRUE. The bare string is cast by the boolean schema, so the obvious command does the opposite of what was asked and config:get then prints moderation: true as if nothing happened. 0 or --input-format=yaml false work.
  • drush key:delete is broken on Drush 13 (Not enough arguments (missing: "options"), Drush 13.7.4), so rollback uses drush config:delete key.key.<id> with the no-cascade caveat spelled out.
  • Default models are not set on the CLI path. OpenAiConfigForm::submitForm() calls setDefaultModels(); config:set does not. Skip it and the provider is authenticated but nothing routes to it. The skill also notes the plugin ID is openai, not the module name.

Testing steps

Use a scratch/local site. Steps 1–4 need no OpenAI account. Step 8 makes one live read-only API call, so it needs a real key — skip it if you would rather not.

Commands below assume ddev; drop the ddev prefix otherwise.

Setup

  1. Check out the MR branch into an existing site that has ai, ai_provider_openai and key enabled:

    ddev drush pm:list --status=enabled --filter=key --fields=name,status

  2. Record the current state so you can restore it afterwards:

    ddev drush key:list
ddev drush config:get ai_provider_openai.settings --format=yaml

1. The skill is discoverable and readable

  1. Confirm .agents/skills/setup-provider/SKILL.md exists, has YAML frontmatter with name: setup-provider and a description, and that the Step 0 gate appears before any command in the document.

2. Redacted variable check (no secret needed)

  1. Set a throwaway variable and run the skill's Step 3 checks. The value must never appear in the output:

    ddev exec 'SKILL_TEST_KEY=sk-test-abcdef123456 sh -c '"'"'if [ -n "${SKILL_TEST_KEY:-}" ]; then echo "set (${#SKILL_TEST_KEY} chars)"; else echo "NOT set"; fi'"'"''

    Expected: set (20 chars) — a count, not the value.

  2. Run the persistence check, which must print names only:

    ddev exec 'printenv | cut -d= -f1 | grep -cx IS_DDEV_PROJECT'
ddev exec 'printenv | cut -d= -f1 | grep -cx SKILL_TEST_KEY'

    Expected: 1 then 0 (exit status 1). The second result is the point — the variable passed the length check in step 4 but is not in the persistent environment, which is what the skill tells you to catch.

3. Key creation refuses the secret on the command line

  1. Try to pass a value to an env-provider key:

    ddev drush key:save zzz_test 'somevalue' --label="T" --key-type=authentication \
  --key-provider=env \
  --key-provider-settings='{"env_variable":"SKILL_TEST_KEY","base64_encoded":false,"strip_line_breaks":true}' \
  --key-input=none -y

    Expected: fails with "The selected key provider does not support setting a key value." No key is created.

  2. Create it the way the skill does — no value argument:

    ddev drush key:save zzz_test --label="Skill Test" --key-type=authentication \
  --key-provider=env \
  --key-provider-settings='{"env_variable":"SKILL_TEST_KEY","base64_encoded":false,"strip_line_breaks":true}' \
  --key-input=none -y
ddev drush config:get key.key.zzz_test --format=yaml

    Expected: silent success; the entity has a populated uuid, key_provider: env, env_variable: SKILL_TEST_KEY, and no key value anywhere in the output.

4. Redacted resolution check, both outcomes

  1. With the variable present:

    ddev exec 'SKILL_TEST_KEY=sk-test-abcdef123456 drush php:eval '"'"'$k = \Drupal::service("key.repository")->getKey("zzz_test"); $v = $k ? $k->getKeyValue() : NULL; print $v ? "RESOLVED length=" . strlen($v) : "EMPTY";'"'"''

    Expected: RESOLVED length=20.

  2. Without it:

    ddev drush php:eval '$k = \Drupal::service("key.repository")->getKey("zzz_test"); $v = $k ? $k->getKeyValue() : NULL; print $v ? "RESOLVED length=" . strlen($v) : "EMPTY";'

    Expected: EMPTY — the case the skill blocks on before Step 6.

5. The documented Drush behaviours

  1. Boolean handling. Note the current value first, then:

    ddev drush config:set ai_provider_openai.settings moderation false -y
ddev drush php:eval 'var_export(\Drupal::config("ai_provider_openai.settings")->get("moderation"));'

    Expected: true — the write did the opposite of what it says. Now:

    ddev drush config:set --input-format=yaml ai_provider_openai.settings moderation false -y
ddev drush php:eval 'var_export(\Drupal::config("ai_provider_openai.settings")->get("moderation"));'

    Expected: false. Restore your original value with --input-format=yaml.

  2. key:delete on Drush 13:

    ddev drush key:delete zzz_test -y

    Expected: Not enough arguments (missing: "options"). The rollback path in the skill uses config:delete instead:

    ddev drush config:delete key.key.zzz_test -y
ddev drush key:list

6. Misleading downstream error (optional, no valid key needed)

  1. Recreate zzz_test per step 7 with the variable absent, point the provider at it, and query models:

    ddev drush config:set ai_provider_openai.settings api_key zzz_test -y
ddev drush php:eval 'try { $p = \Drupal::service("ai.provider")->createInstance("openai"); print "OK models=" . count($p->getConfiguredModels("chat")); } catch (\Throwable $e) { print get_class($e) . ": " . $e->getMessage(); }'

    Expected: a logged "Could not load the OpenAI API key" followed by OpenAI\Exceptions\ErrorException: Missing bearer authentication in header — the symptom that does not name the cause.

  2. Restore the previous api_key from step 2 and delete zzz_test via config:delete.

7. Full run through the skill (needs a real key)

  1. Put a real key in the container environment:

    ddev config --web-environment-add=OPENAI_API_KEY=sk-your-key-here
ddev restart

  2. Have an agent run the skill end to end. Verify it: asks before doing anything; never prints the key; creates the key; writes only the key ID to api_key; sets default models with provider_id: openai; and reports the variable name plus a character count.

  3. Confirm connectivity:

    ddev drush php:eval 'try { $p = \Drupal::service("ai.provider")->createInstance("openai"); print "OK models=" . count($p->getConfiguredModels("chat")); } catch (\Throwable $e) { print get_class($e) . ": " . $e->getMessage(); }'

    Expected: OK models=<n>.

  4. Confirm the web process agrees: open /admin/config/ai/providers/openai, select the new key, and press Save configuration. It should save without a validation error — that runs the form's real key-value, connectivity and rate-limit checks inside PHP-FPM.

  5. Confirm the secret is not in config:

    ddev drush config:export --destination=/tmp/key-audit -y
cat /tmp/key-audit/key.key.<your-id>.yml

    Expected: the variable name under env_variable and no credential.

Cleanup

  1. Restore the api_key, moderation and key list you recorded in step 2, and remove any zzz_test keys and the throwaway variable.

Merge request reports