Responses (1)

Sometimes needs are so variable they can’t be handled by passing options, you need to allow the user to customize the code itself.

This article presents the rationale and implementation of the plugin architecture we used in the next evolution of the FeathersJS local authentication management repo.

Where do you draw the line?

People sometimes want to left-pad a number to form a fixed length string, to convert 17 to 0017. Repos like left-pad have a clear focus but they don’t handle closely related needs which people may have, such as:

• I’d like to show 2 decimal places.
• I’d like the shown decimal places to be rounded, or not.
• I’d like digit separators to be shown, e.g. 1,234.
• I’d like the digit and decimal separator to be variable, e.g. 1 234,56.
• I’d like a trailing decimal separator to be shown, e.g. 1234. with no decimals.
• I’d like a variable curreny symbol to be shown, left-justified or floating, e.g. $1234 or $ 1234.

Imagine getting these requests if you’re the author of left-pad. You wrote a repo to address a specific common need you had, and you published it to help others. Now you are being asked to spend time implementing new features which you yourself will not use.

Frankly this is not supportable for long. Adrian Holovaty, one of the Benevolent Dictators for Life of Django, writing about his retirement from Django, mentioned there wasn’t much need for internationalization into German where he worked at theLawrence Journal-World daily newspaper published in Lawrence, Kansas, United States.

I’ll be waiting on the other side

Repos may be most successful when they have a specific focus and address that well. But what is the user with a slightly different need to do?

If he’s lucky, he can wrap the repo and implement his specialized needs in the wrapper. A leading currency symbol could be implemented with

const leftPad = require('left-pad');module.exports = (amt, width, symbol = '$') =>
  `${symbol}${leftPad(value, width - 1})`;

Most of time however some code in the repo has to be modified, or some additional code needs to be added. This means you have to fork the repo, creating your own personal version of it, and customize that copy. This comes with its own downsides:

• Your fork will not automatically benefit from code corrections and enhancements made to the original repo.
• Other developers are not familiar with your forked version and may even get confused by it.

Think outside the box

Some repos are involved with issues which are variable and elusive by their very nature. Authentication is one such area. These repos can expect to get more feature requests than other types of repos, and these requests will be for reasonable use cases.

We faced this situation while rewriting feathers-plus/ authentication-local-management (a.k.a. a-l-m) for FeathersJS. This was the third major refactoring of the repo (callbacks to Promises to async/await) so we had a good understanding of the requirements. We also had experience with specialized use cases users were addressing.

We decided to introduce some techniques which made the new repo more customization-friendly, while maintaining its focus.

Linear perspective

We are not talking about a little flexibility here and a little there.Some background may help you appreciate the scope of our issue.

FeathersJS is an open source GraphQL, REST and realtime API layer for modern applications. It has 10k GitHub stars; it’s one of the 8 backend frameworks selected by the State of JavaScript survey in 2017 and one of the 6 in 2018.

Feathers’ a-l-m uses email and SMS to

• invite a potential new user recommended by a current user
• verify a new user
• verify a new user if the original verification message was lost
• implement two factor authentication (2FA) on sign in
• reset the password if its forgotten
• verify the sign in if from a new device
• verify the sign in if the last sign in was too long ago
• notify of a password change
• verify a change to the email, phone number, PIN, badge, etc.

The package works in conjunction with the frontend and with push providers. One of the simpler scenarios it handles is resetting a forgotten password using an SMS message. The sequence of events is shown below, where a-l-m is directly involved with the red interactions.

You can sense there is a lot of room for specialized use cases in such a repo, including:

• The new password must satisfy criteria involving length, lower and upper-case letters, digits, and special symbols.
• Successful resets must be recorded.
• Changed passwords must be added to a trail of out-of-date passwords so a user cannot reuse an old password.
• A new password cannot be one of the user’s old passwords.
• Specifying the types of notifications attempted and in which order.
• This list is not exhaustive.

Ways of stepping over the line

We want users to have a chance of implementing customizations we haven’t yet heard of.This requires:

• Modification of code presently in the repo.
• Addition of code to the repo.

We’ve already discounted forks and will now consider options and plugins.

We decided to introduce some techniques which made the new repo customization-friendly, while maintaining its original focus.

Options gone wild

Larger repos often support configuration options, and they are great for many things, and a-l-m certainly uses options:

const optionsDefault = {
  app: null, // Value set during configuration.
service: '/users',path: 'authManagement',
 emailField: 'email',
 dialablePhoneField: 'dialablePhone',
  passwordField: 'password',
longTokenLen: 15,
 shortTokenLen: 6,
  shortTokenDigits: true,
  resetDelay: 1000 * 60 * 60 * 2,delay: 1000 * 60 * 60 * 24 * 5,identifyUserProps: ['email', 'dialablePhone'],
  actionsNoAuth: [
    'resendVerifySignup', 'verifySignupLong', 'verifySignupShort',
    'sendResetPwd', 'resetPwdLong', 'resetPwdShort',
  ],
  ownAcctOnly: true,
  plugins: null,
};

Options are not as good for stringing together a piece of code. Consider these Feathers service calls:

// Feathers service calls to get record 'id' in the users DB
// as server
user = await users.get(id);
// as an unauthenticated client in a REST request
user = await users.get(id, { provider: 'rest' });
// as an authenticated client
user = await users.get(id, { users: authenticatedClient });

How a service call is made in the repo may be very important because middleware running before and after the actual database call may behave differently depending on the circumstances. Options are not an ideal mechanism for this syntactic variance because you will ultimately need an option for every possible syntactic unit in the code. Frankly the options will be out of control.

Here’s what happened when we tried using options for sections of code. The original code was:

const users = await usersService.find({
  { query: identifyUser }, provider
});

const user3 = await usersService.patch(
  user2[usersServiceIdName], {
     resetExpires: user2.resetExpires,
     resetToken: user2.resetToken,
     resetShortToken: user2.resetShortToken,
  }
);

The options based code became:

const users = await options.customizeCalls.sendResetPwd.find(
  usersService, { { query: identifyUser }, provider }
);

const user3 = await options.customizeCalls.sendResetPwd.patch(
  usersService, user2[usersServiceIdName], {
     resetExpires: user2.resetExpires,
     resetToken: user2.resetToken,
     resetShortToken: user2.resetShortToken,
  }
);// The default options contain the original code:
const optionsCustomizedCalls = {
sendResetPwd: {
find: async (usersService, params) =>
      await usersService.find(params),
patch: async (usersService, id, data, params = {}) =>
      await usersService.patch(id, data, params),
},
}:

The user can now replace critical sections of code in the repo, so that’s an improvement. However issues remain. First, a fair number of the possible customizations involve adding a line of code before or after a key piece of code in the repo. This would lead to awkward looking situations where the customized options kept having to repeat the code in the option being replaced.

// Default option
const optionsCustomizedCalls = {
  sendResetPwd: {
    patch: async (usersService, id, data, params) =>
await usersService.patch(id, data, params),
},
}:// Replacement option
const optionsCustomizedCalls = {
sendResetPwd: {
   patch: async (usersService, id, data, params) => {
 await usersService.patch(id, data, params);
      // Add password to list of user's passwords
      // so it cannot be reused
      await usersPasswordsService.create({ id, data.password });
    }
  },
}:

Second, if the user wants a-l-m to support a new command — such as sending an SMS message whenever a sign-in was detected from a new device — the user would have to override the entire command dispatching code shown below.

try {
switch (data.action) {
case 'checkUnique':
      return await checkUnique(
        options, data.value, data.ownId || null, data.meta || {},
        data.authUser, data.provider);
case 'resendVerifySignup':
      return await resendVerifySignup(
        options, data.value, data.notifierOptions,
        data.authUser, data.provider);
case 'verifySignupLong':
      return await verifySignupWithLongToken(
        options, data.value,
        data.authUser, data.provider);
case 'verifySignupShort':
      return await verifySignupWithShortToken(
        options, data.value.token, data.value.user,
        data.authUser, data.provider);
case 'sendResetPwd':
      return await sendResetPwd(
        options, data.value, data.notifierOptions,
        data.authUser, data.provider);
case 'resetPwdLong':
      return await resetPwdWithLongToken(
        options, data.value.token, data.value.password,
        data.authUser, data.provider);
case 'resetPwdShort':
      return await resetPwdWithShortToken(
        options, data.value.token, data.value.user,
        data.value.password,
        data.authUser, data.provider);
case 'passwordChange':
      return await passwordChange(
        options, data.value.user, data.value.oldPassword,
        data.value.password,
        data.authUser, data.provider);
case 'identityChange':
      return await identityChange(
        options, data.value.user, data.value.password,
        data.value.changes,
        data.authUser, data.provider);
default:
      return Promise.reject(
        new errors.BadRequest(`Action '${data.action}' is invalid.`,
          { errors: { $className: 'badParams' } }
        )
      );
  }
} catch (err) {
  return options.catchErr(err, options, data);
}

That customized code could not take advantage of any changes made to the repo, including the introduction of new commands.

Finally, using options like this looked fragile. We should create an abstraction if we are going to often depend on something like this. That’s how we evolved into using plugins.

Plugins

A plugin is a software component that adds a specific feature to an existing program. The popular JavaScript linting utility eslint has a pluggable design. WordPress has over 50,000 plugins.

Plugins are executable pieces of code. For our purposes here we will consider even a single line of code as a potential plugin.

Our repo looks very similar whether its using options

const users = await options.customizeCalls.sendResetPwd.find(
  usersService, { { query: identifyUser }, provider }
);

const user3 = await options.customizeCalls.sendResetPwd.patch(
  usersService, user2[usersServiceIdName], {
     resetExpires: user2.resetExpires,
     resetToken: user2.resetToken,
     resetShortToken: user2.resetShortToken,
  }
);

or plugins

const users = await plugins.run('sendResetPwd.find', {
  usersService, params: { { query: identifyUser }, provider }
});const user4 = await plugins.run('sendResetPwd.patch', {
  usersService, id: user3[usersServiceIdName], data: {
    resetExpires: user2.resetExpires,
    resetToken: user2.resetToken,
    resetShortToken: user2.resetShortToken,
  },
});

But what happens behind the scenes in a plugin is very different.

We should create an abstraction if we are going to often depend on something like this. That’s how we evolved to using plugins.

plugin-scaffolding

We published our plugin infrastructure as feathers-plus/plugin-scaffolding. Its plugins are characterized by:

Arrays

A plugin is an array of async functions which are executed sequentially when the plugin is triggered. The reduced result is returned as the value of the plugin, much like how the Javascript Array.reducemethod works.

// Plugin function
async (accumulator, data, pluginsContext, pluginContext) =>
  accumulator === undefined ? 1: accumulator + 1;

Order

When you register a new plugin for a particular trigger, you indicate if it should be run before or after the existing plugins for that trigger, or if it should replace them.

plugins.register({
name: 'sendResetPwd.find',
desc: 'sendResetPwd.find - default plugin',
version: '1.0.0',
trigger: 'sendResetPwd.find',
position: 'clear', // or 'before', 'after'
run: async (accumulator, { usersService, params}, pluginsContext,
   pluginContext) => await usersService.find(params),
});

A repo may therefore register default plugins for a trigger and the user can add additional plugins before or after the default one. The user may also replace the default.

Turtles all the way down

A plugin may itself call plugins. A plugin handling a new command may internally call plugins for smaller pieces of code. The user customizes at the most convenient level.

Setup and teardown

Each plugin may have a function which runs before the plugins are ready for use. Let’s say your plugin is adding a new command. The plugin’s setup may add defaults to the repo’s options for this new capability. The user then overrides just the ones he wants.

plugins.register({
trigger: 'customCommand',
setup: async (pluginsContext, pluginContext) =>
    merge(pluginsContext.options, { /* new default options */ }),
teardown: async (pluginsContext, pluginContext) => {},
run: async (...) {...),
});

Similarly there is a teardown function.

Context

All plugins share a context for communication between themselves. This context would normally contain the parent repo’s options and access to all the plugins.

plugins.register({
trigger: 'customCommand',
setup: async (pluginsContext, pluginContext) =>
    // Add custom props to repo's options
    merge(pluginsContext.options, { bar: 'baz }),
run: async (accumulator, data, pluginsContext, pluginContext) => {
    // This plugin calls another
    const plugins = pluginsContext.plugins;
    let users = await plugins('faz', { ... });
    // Set a flag which only other plugins for this trigger can see
    pluginContext.foo = true;
  },
});

Plugin initialization

Initializing the plugins is straightforward. Here the custom plugins are passed in userOptions.plugins.

const Plugins = require('@feathers-plus/plugin-scaffolding');
const pluginsDefault = require('./plugins-default');const defaultOptions: { ... }; // repos's default optionsfunction constructor(userOptions) { // user's options  // Load plugins. They may add additional default options.
const pluginsContext = { options: defaultOptions };
const plugins = new Plugins(pluginsContext);
plugins.register(pluginsDefault);

if (userOptions.plugins) {
plugins.register(userOptions.plugins);
  }

(async function() {
await plugins.setup();
  }());

// Get final options
  options = pluginsContext.options =  Object.assign(
    defaultOptions, userOptions, { plugins }
  );};

Command dispatching

A repos’s command dispatching can now dispatch new, customized commands automatically, avoiding the problems we had with the options-based approach.

const trigger = data.action;if (!plugins.has(trigger)) { // Check plugin exists.
  return Promise.reject(
    new errors.BadRequest(`Action '${trigger}' is invalid.`,
      { errors: { $className: 'badParams' } }
    )
  );
}try {
return await plugins.run(trigger, data);
} catch (err) {
  return await plugins.run('catchError', err);
}

… what happens behind the scenes in a plugin is very different.

Emergent benefits

Plugins have provided some unexpected benefits.

Fewer options

Previously a-l-m had a notifier option whose value was a function to call whenever a notification was to be sent. a-l-m now just runs the notifier plugin.

This shows plugins can replace some types of options.

Code insertion points

We can predict some places where users may want to add code. Even if we ourselves don’t have anything we want to do there, we can still call a no-op plugin. The user can override that plugin to execute their code.

run: async(accumulator, data, pluginsContext, pluginContext)
=> accumulator || data

Plugin ecosystem

Users can publish their plugins for others to use. This may be especially useful for new commands.

We’re considering encouraging that by adding a plugin-ecosystem folder to a-l-m or by publishing an authentication-local-management-ecosystem repo. However doing either may suggest we have a responsibility to maintain those contributions.

When the plug meets the socket

a-l-m calls a notifier whenever the user might want to send a push notification. This call is made by running the notifier plugin:

await plugins.run('notifier', {
type: 'sendResetPwd', // Type of notification.
sanitizedUser: user3, // Contains all info for push.
notifierOptions,      // Customization options.
});

The raw default plugin is a no-op:

run: async (accum, { type, sanitizedUser, notifierOptions },
    { options }, pluginContext) => return sanitizedUser

The a-l-m test suite confirms the notifier is called whenever necessary by overriding the notifier plugin:

// Setup test that 'sendResetPwd' calls the notifier
beforeEach(async () => {
stack = [];  app.configure(authLocalMgnt({
    longTokenLen: 15,
    shortTokenLen: 6,
    shortTokenDigits: true,
plugins: [{
      trigger: 'notifier',
      position: 'clear',
run: async (accum, { type, sanitizedUser, notifierOptions },
          { options }, pluginContext) => {        // Cache notifier's input and output
stack.push({
args: clone([type, sanitizedUser, notifierOptions]),
result: clone(sanitizedUser),
});return sanitizedUser;
      },
}],
  }));
});

and checking the parameters the notifier was called with:

// Test notifier is called
it('Notifier is called', async function () {
  const result = await authLocalMgntService.create({
    action: 'sendResetPwd',
    value: { email: 'b' },
    notifierOptions: { transport: 'sms' }
  });  // ...
const actual= stack[0].args;
  assert.deepEqual(actual, expected);
});

The test confirms the notifier plugin had been replaced.

Pros and Cons of our plugins

Plugins have addressed key needs. Users can now:

• Add new commands for entirely new services — such as sending an SMS message whenever a sign in was detected from a new device — by creating a new plugin.
• These new commands automatically call their service because of the nature of the new routing.
• Replace an entire existing service by overriding the service’s plugin.
• Disable an existing service by overriding its service with a no-op.
• Run specialized code before and/or after each service.
• Change some of the critical lines in a service.
• Run specialized code before and/or after these critical lines.
• Inject new code in a service at code insertion points.

• The resulting repo has a great deal of flexibility resulting not from a set of options, but from being able to change or add code.
• Adding a new or changing an exising service results in reasonably clean code.

• You are exposed to all the details of the service when you change a plugin called within that service. The alternative, arguably worse, is to fork the project.

• The code is harder to understand because it calls plugins.
• You have to identify which plugin functions will be run (default and custom) before you know what the code does.

In conclusion

This article is part of a series of articles on Feathers’ user management. In the next article we’ll discuss how to configure a-l-m.

So we’re not done yet. Subscribe to The Feathers Flightpath publication to be informed of the coming articles.

As always, feel free to join Feathers Slack to join the discussion or just lurk around.

Note: The npm publication feathers-plus/authentication-local-management repo discussed in this article was known as npm publication feathers-authentication-management in its two previous major versions.