Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletions docs/pages/run/validator-management/vc-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@ To import a validator JSON keystore that was created via one of the methods desc

Inside the keystore JSON file, you should have an [EIP-2335 keystore file](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-2335.md#json-schema).

You will also need the passphrase used the encrypt the keystore. This can be specified interactively, or provided in a plaintext file.
You will also need the passphrase used to encrypt the keystore. This can be entered interactively, provided in a shared plaintext file, or provided as per-keystore plaintext files.

#### Option 1: Import Keys To Lodestar's Keystores Folder

You can load the keys into the keystore folder using the [`validator import`](../validator-management/validator-cli.md#validator-import) command. There are two methods for importing keystores:
You can load the keys into the keystore folder using the [`validator import`](../validator-management/validator-cli.md#validator-import) command. There are three methods for importing keystores:

_Interactive passphrase import_

Expand All @@ -44,24 +44,36 @@ _Plaintext passphrase file import_
./lodestar validator import --importKeystores ./validator_keys --importKeystoresPassword ./password.txt
```

_Per-keystore plaintext passphrase file import_

```bash
./lodestar validator import --importKeystores ./validator_keys --importKeystoresPasswords ./passwords
```

:::info
The interactive passphrase import method will prompt every keystore in the `validator_keys` folder for import and will ask for the individual password for each keystore. **This method will allow you to import multiple keystores with different passwords.**
The interactive passphrase import method will prompt once for a shared password and use it for every keystore found in the `validator_keys` folder.

The plaintext passphrase file import method will allow you to import all keystores in the `validator_keys` folder encrypted with the same password contained in `password.txt` for efficiency.

The per-keystore plaintext passphrase file import method expects one file per validator in `./passwords`, named `0x<validator-public-key-hex>.txt`.
:::

Once imported with either method, these keystores will be automatically loaded when you start the validator. To list the imported keystores, use the [`validator list`](./validator-cli.md#validator-list) command.
Once imported with any of these methods, these keystores will be automatically loaded when you start the validator. To list the imported keystores, use the [`validator list`](./validator-cli.md#validator-list) command.

---

#### Option 2: Import Keys When Starting the Validator

To import keys when you start the validator specify the [`--importKeystores`](./validator-cli.md#--importkeystores) and [`--importKeystoresPassword`](./validator-cli.md#--importkeystorespassword) flags with the [`validator`](./validator-cli.md#base-validator-command) command:
To import keys when you start the validator, specify the [`--importKeystores`](./validator-cli.md#--importkeystores) flag on the [`validator`](./validator-cli.md#base-validator-command) command. To avoid the interactive prompt, also pass either [`--importKeystoresPassword`](./validator-cli.md#--importkeystorespassword) for a shared password file or [`--importKeystoresPasswords`](./validator-cli.md#--importkeystorespasswords) for per-keystore password files:

```bash
./lodestar validator --importKeystores ./validator_keys --importKeystoresPassword ./password.txt
```

```bash
./lodestar validator --importKeystores ./validator_keys --importKeystoresPasswords ./passwords
```

:::warning
If you import keys using `--importKeystores` at runtime (Option 2) any keys loaded to the keystores folder from Option 1 will be ignored.
:::
Expand Down
29 changes: 18 additions & 11 deletions packages/cli/src/cmds/validator/import.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,21 @@ import {YargsError, getPubkeyHexFromKeystore} from "../../util/index.js";
import {PersistedKeysBackend} from "./keymanager/persistedKeys.js";
import {IValidatorCliArgs, validatorOptions} from "./options.js";
import {getAccountPaths} from "./paths.js";
import {importKeystoreDefinitionsFromExternalDir, readPassphraseOrPrompt} from "./signers/importExternalKeystores.js";
import {readKeystoreDefinitionsFromArgs} from "./signers/importExternalKeystores.js";

type ValidatorImportArgs = Pick<IValidatorCliArgs, "importKeystores" | "importKeystoresPassword">;
type ValidatorImportArgs = Pick<
IValidatorCliArgs,
"importKeystores" | "importKeystoresPassword" | "importKeystoresPasswords"
>;

const {importKeystores, importKeystoresPassword} = validatorOptions;
const {importKeystores, importKeystoresPassword, importKeystoresPasswords} = validatorOptions;

export const importCmd: CliCommand<ValidatorImportArgs, IValidatorCliArgs & GlobalArgs> = {
command: "import",

describe:
"Imports one or more EIP-2335 keystores into a Lodestar validator client directory, \
requesting passwords interactively. The directory flag provides a convenient \
using either a shared password or per-keystore password files. The directory flag provides a convenient \
method for importing a directory of keys generated by the eth2-deposit-cli \
Ethereum Foundation utility.",

Expand All @@ -27,22 +30,28 @@ Ethereum Foundation utility.",
command: "validator import --network hoodi --importKeystores $HOME/staking-deposit-cli/validator_keys",
description: "Import validator keystores generated with the Ethereum Foundation Staking Launchpad",
},
{
command:
"validator import --network hoodi --importKeystores $HOME/staking-deposit-cli/validator_keys --importKeystoresPasswords $HOME/staking-deposit-cli/passwords",
description: "Import validator keystores with one password file per validator public key",
},
],

// Note: re-uses `--importKeystores` and `--importKeystoresPassword` from root validator command options
// Note: re-uses `--importKeystores`, `--importKeystoresPassword`, and `--importKeystoresPasswords` from root validator command options

options: {
importKeystores: {
...importKeystores,
requiresArg: true,
},
importKeystoresPassword,
importKeystoresPasswords,
},

handler: async (args) => {
const {network} = getBeaconConfigFromArgs(args);

// This command takes: importKeystores, importKeystoresPassword
// This command takes: importKeystores, importKeystoresPassword, importKeystoresPasswords
//
// - recursively finds keystores in importKeystores
// - validates keystores can decrypt
Expand All @@ -52,12 +61,10 @@ Ethereum Foundation utility.",
throw new YargsError("Must set importKeystores");
}

// Collect same password for all keystores
// If importKeystoresPassword is not provided, interactive prompt for it

const keystoreDefinitions = importKeystoreDefinitionsFromExternalDir({
const keystoreDefinitions = await readKeystoreDefinitionsFromArgs({
keystoresPath: args.importKeystores,
password: await readPassphraseOrPrompt(args),
importKeystoresPassword: args.importKeystoresPassword,
importKeystoresPasswords: args.importKeystoresPasswords,
});

if (keystoreDefinitions.length === 0) {
Expand Down
10 changes: 10 additions & 0 deletions packages/cli/src/cmds/validator/options.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ export type IValidatorCliArgs = AccountValidatorArgs &

importKeystores?: string[];
importKeystoresPassword?: string;
importKeystoresPasswords?: string;
disableKeystoresThreadPool?: boolean;

"http.requestWireFormat"?: string;
Expand Down Expand Up @@ -303,6 +304,15 @@ export const validatorOptions: CliCommandOptions<IValidatorCliArgs> = {
alias: ["passphraseFile"], // Backwards compatibility with old `validator import` cmd
description: "Path to a file with password to decrypt all keystores from `importKeystores` option",
defaultDescription: "./password.txt",
conflicts: ["importKeystoresPasswords"],
type: "string",
},

importKeystoresPasswords: {
description:
"Path to a directory containing per-keystore password files named `0x<validator-public-key-hex>.txt` for `importKeystores`",
defaultDescription: "./passwords/0x<validator-public-key-hex>.txt",
conflicts: ["importKeystoresPassword"],
type: "string",
},

Expand Down
104 changes: 93 additions & 11 deletions packages/cli/src/cmds/validator/signers/importExternalKeystores.ts
Original file line number Diff line number Diff line change
@@ -1,28 +1,110 @@
import fs from "node:fs";
import path from "node:path";
import inquirer from "inquirer";
import {readPassphraseFile, recursiveLookup} from "../../../util/index.js";
import {Keystore} from "@chainsafe/bls-keystore";
import {YargsError, getPubkeyHexFromKeystore, readPassphraseFile, recursiveLookup} from "../../../util/index.js";
import {LocalKeystoreDefinition} from "../keymanager/interface.js";

type ImportKeystoreDefinitionsFromExternalDirArgs =
| {
keystoresPath: string[];
password: string;
}
| {
keystoresPath: string[];
passwordsDir: string;
};

/**
* Imports keystores from un-controlled directories provided by the user.
* This directories are expected to contain other files, so some filtering in done to improve UX.
* @param args
*/
export function importKeystoreDefinitionsFromExternalDir(args: {
export function importKeystoreDefinitionsFromExternalDir(
args: ImportKeystoreDefinitionsFromExternalDirArgs
): LocalKeystoreDefinition[] {
if (!("password" in args)) {
assertValidPasswordsDir(args.passwordsDir);
}

const keystorePaths = getVotingKeystorePaths(args.keystoresPath);

if ("password" in args) {
return keystorePaths.map((keystorePath) => ({
keystorePath,
password: args.password,
}));
}

return keystorePaths.map((keystorePath) => ({
keystorePath,
password: readPassphraseFile(getPasswordFilepathForKeystore(keystorePath, args.passwordsDir)),
}));
}

export async function readKeystoreDefinitionsFromArgs(args: {
keystoresPath: string[];
password: string;
}): LocalKeystoreDefinition[] {
importKeystoresPassword?: string;
importKeystoresPasswords?: string;
}): Promise<LocalKeystoreDefinition[]> {
if (args.importKeystoresPasswords) {
return importKeystoreDefinitionsFromExternalDir({
keystoresPath: args.keystoresPath,
passwordsDir: args.importKeystoresPasswords,
});
}

return importKeystoreDefinitionsFromExternalDir({
keystoresPath: args.keystoresPath,
password: await readPassphraseOrPrompt(args),
});
}

function getVotingKeystorePaths(keystoresPath: string[]): string[] {
const allFiles: string[] = [];

for (const keystorePath of args.keystoresPath) {
for (const keystorePath of keystoresPath) {
assertValidKeystoresPath(keystorePath);
recursiveLookup(keystorePath, allFiles);
}

return allFiles
.filter((filepath) => isVotingKeystore(filepath))
.map((keystorePath) => ({
keystorePath,
password: args.password,
}));
return allFiles.filter((filepath) => isVotingKeystore(filepath));
}

function assertValidKeystoresPath(keystorePath: string): void {
if (!fs.existsSync(keystorePath)) {
throw new YargsError(`importKeystores must point to an existing file or directory: ${keystorePath}`);
}
}

function assertValidPasswordsDir(passwordsDir: string): void {
if (!fs.existsSync(passwordsDir)) {
throw new YargsError(`importKeystoresPasswords must point to an existing directory: ${passwordsDir}`);
}

if (!fs.statSync(passwordsDir).isDirectory()) {
throw new YargsError(`importKeystoresPasswords must point to a directory: ${passwordsDir}`);
}
}

function getPasswordFilepathForKeystore(keystorePath: string, passwordsDir: string): string {
let pubkeyHex: string;
try {
const keystore = Keystore.parse(fs.readFileSync(keystorePath, "utf8"));
pubkeyHex = getPubkeyHexFromKeystore(keystore);
} catch (e) {
throw new YargsError(`Failed to read keystore ${keystorePath}: ${e instanceof Error ? e.message : String(e)}`);
}

const passwordFilepath = path.join(passwordsDir, `${pubkeyHex.toLowerCase()}.txt`);

if (!fs.existsSync(passwordFilepath)) {
throw new YargsError(
`No password file found for keystore ${keystorePath}. Expected password file ${passwordFilepath}`
);
}

return passwordFilepath;
}

export async function readPassphraseOrPrompt(args: {importKeystoresPassword?: string}): Promise<string> {
Expand Down
9 changes: 5 additions & 4 deletions packages/cli/src/cmds/validator/signers/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,15 @@ import {decryptKeystoreDefinitions} from "../keymanager/decryptKeystoreDefinitio
import {PersistedKeysBackend} from "../keymanager/persistedKeys.js";
import {IValidatorCliArgs} from "../options.js";
import {getAccountPaths} from "../paths.js";
import {importKeystoreDefinitionsFromExternalDir, readPassphraseOrPrompt} from "./importExternalKeystores.js";
import {readKeystoreDefinitionsFromArgs} from "./importExternalKeystores.js";

const KEYSTORE_IMPORT_PROGRESS_MS = 10000;

/**
* Options processing hierarchy
* --interopIndexes
* --fromMnemonic, then requires --mnemonicIndexes
* --importKeystores, then requires --importKeystoresPassword
* --importKeystores, then supports --importKeystoresPassword or --importKeystoresPasswords
* --externalSigner.fetch, then requires --externalSigner.urls
* --externalSigner.pubkeys, then requires --externalSigner.urls
* else load from persisted
Expand Down Expand Up @@ -76,9 +76,10 @@ export async function getSignersFromArgs(

// Import JSON keystores and run
if (args.importKeystores) {
const keystoreDefinitions = importKeystoreDefinitionsFromExternalDir({
const keystoreDefinitions = await readKeystoreDefinitionsFromArgs({
keystoresPath: args.importKeystores,
password: await readPassphraseOrPrompt(args),
importKeystoresPassword: args.importKeystoresPassword,
importKeystoresPasswords: args.importKeystoresPasswords,
});

const needle = showProgress({
Expand Down
44 changes: 44 additions & 0 deletions packages/cli/test/e2e/importFromFsDirect.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,47 @@ describe("import from fs same cmd as validate", () => {
await stopValidator();
});
});

describe("import from fs same cmd as validate with per-keystore passwords", () => {
vi.setConfig({testTimeout: 30_000});

const dataDir = path.join(testFilesDir, "import-and-validate-multi-password-test");
const importFromDir = path.join(dataDir, "eth2.0_deposit_out");
const passwordsDir = path.join(dataDir, "passwords");

const passphrases = ["AAAAAAAA0000000000", "BBBBBBBB1111111111"];
const keyCount = passphrases.length;
const pubkeys = cachedPubkeysHex.slice(0, keyCount);
const secretKeys = cachedSeckeysHex.slice(0, keyCount);

beforeAll(async () => {
rimraf.sync(dataDir);
rimraf.sync(importFromDir);
rimraf.sync(passwordsDir);

const keystoresStr = await Promise.all(
secretKeys.map(async (secretKey, index) => {
const [keystoreStr] = await getKeystoresStr(passphrases[index], [secretKey]);
return keystoreStr;
})
);

fs.mkdirSync(importFromDir, {recursive: true});
fs.mkdirSync(passwordsDir, {recursive: true});

for (let i = 0; i < keyCount; i++) {
fs.writeFileSync(path.join(importFromDir, `keystore_${i}.json`), keystoresStr[i]);
fs.writeFileSync(path.join(passwordsDir, `${pubkeys[i]}.txt`), passphrases[i]);
}
});

it("run 'validator' with per-keystore password files and check keys are loaded", async () => {
const {keymanagerClient, stopValidator} = await startValidatorWithKeyManager(
[`--importKeystores=${importFromDir}`, `--importKeystoresPasswords=${passwordsDir}`],
{dataDir, logPrefix: "case-3"}
);

await expectKeys(keymanagerClient, pubkeys, "Wrong listKeys response data");
await stopValidator();
});
});
Loading
Loading