Hi,

We have noticed our instances with a127 to be running out of memory while performing load tests. This is occuring when the a127 proxy reports errors to client. I have not done throughout testing for all possible scenarios of errors, so more investigations for memory leaks should be made.

Found memory leaks when:

• proxy errors when there is no api_key
• proxy errors when the api_key is invalid
• this is possibly happening on any error when error is returned in custom controller by : next(error)

No memory leaks found in the below scenarios:

• api_key is valid

Suggesting that error messages are not cleaned up properly by V8 engine Garbage Collector.

Reproduction steps:

This has been tested using latest a127-magic and latest volos modules. Our use case is more or less a passthrough proxy app (similar to https://github.com/apigee-127/a127-samples/tree/master/passthrough-proxy-with-keyvalidation)

$ git clone [email protected]:apigee-127/a127-samples.git
$ cd a127-samples/
$ npm install
$ cd passthrough-proxy-with-keyvalidation
$ npm install -g node-inspector
$ node-inspector
$ node --debug app.js
$ ab -c 20 -n 10000 http://localhost:10010/
$ ab -c 20 -n 10000 http://localhost:10010/api_key=VALID_API_KEY

Compare the results of memory heap allocation ie. the one without the valid key does not correctly garbage collect error messages.

Our investigation shows 29.5 MB of allocated memory after initial a127 startup, and 78.3MB after 10k connections made (When no api key is set or api key is invalid). The memory usage stays at this level until the app is restarted. There are no memory leaks when using correct API keys.

The problem persists when using vanilla https://github.com/apigee-127/a127-samples/tree/master/passthrough-proxy-with-keyvalidation and in our own custom controller.

Thanks,
Raf

Hi Team,

as a part node js version upgrade we update a127-magic to latest version. Later when we ran npm audit it shows 4 moderate vulnerabilities. Hence kindly look into this issue and let us know. Bellow are the audit log.

PS C:\Users\mma3069\webStormWorkspace\FADFM-29750\fabric-alerting-api> npm audit

npm audit report

validator <13.7.0
Severity: moderate
Inefficient Regular Expression Complexity in validator.js - GHSA-qgmg-gppg-76g5
fix available via npm audit fix --force
Will install [email protected], which is a breaking change
node_modules/z-schema/node_modules/validator
z-schema 3.6.1 - 4.2.3 || 5.0.0
Depends on vulnerable versions of validator
node_modules/z-schema
swagger-tools >=0.8.7
Depends on vulnerable versions of z-schema
node_modules/swagger-tools
a127-magic >=0.11.0
Depends on vulnerable versions of swagger-tools
node_modules/a127-magic

4 moderate severity vulnerabilities

To address all issues (including breaking changes), run:
npm audit fix --force

Hi,

It will be great if the swagger-ui middleware can be added to the middlewares chain (https://github.com/apigee-127/magic/blob/master/lib/middleware.js#L53) as a config option.

Its already part of the swagger tools so it will be nice to have it out of the box instead of adding it manually.

Thanks,
Boris

Unfortunately, a breaking change had to be made in version 0.7.0 of this module. The new version requires the configuration to be initialized asynchronously. If you have been using the old version, you may address this is one of two ways:

1. You could pin the version of a127-magic in your package.json to "0.6.x".

But a better solution is...

2. Update to the new initialization recipe in your app.js. It's a simple change that will basically look like this:

var a127 = require('a127-magic');
var app = require('express')();

a127.init(function(config) {
  app.use(a127.middleware(config));
  app.listen(process.env.PORT || 10010);
});

(See also: https://github.com/apigee-127/a127/blob/master/project-skeleton/app.js)

This happens when I throw an error in my code. This:

throw 'An error has occurred';

results in a 503, but this:

throw { "message": "An error has occurred" };

results in a 200/OK message.

I don't know if A127 is using errors to send non-error data or if this is just a glitch, but in either case, an exception should surely result in a 5xx response?

It is a particular problem because the Volos cache module throws errors of exactly this form, resulting in our users getting 200/OK messages when actually the server has errored.

Whenever a request has invalid parameters, a127 middleware automatically stops the request flow and send back a response like this:

Error: Parameter (body) is not a valid SomeModel model
... stack trace

We should be able to suppress this stack trace, at least for the production environment.

I could not found any docs on this issue, neither by inspecting the source code.

Please see apigee-127/swagger-tools#327 for more information

The .init function is not following standard callback(error, result) style which makes it impossible to pass back any errors.

Hi,

Added a middleware after the app.use(a127.middleware()) call. But this middleware gets ignored. If I add it before the new middleware is run.

Is there any standard pattern to add a middleware after a127's?

Thanks
José

Add a req.a127.config() method similar to the req.a127.resource() method.

Hi,

The swagger router middleware accepts array of controller paths (#19), but the magic module doesn't support this option as a configuration option to the user (support only one path as string).

As can be seen here, behind the scenes it actually passing array of paths but it expects the 'magic.controllers.controllers' to be a string and when passing an array an exception is thrown.

To make it work, it requires 2 changes:

1. replace this line with:

  if (mainControllers) { 
    if (_.isString(magic.controllers.controllers)) {
        controllers.push(mainControllers);
    } else if (_.isArray(magic.controllers.controllers)) {
        controllers = controllers.concat(mainControllers);
    }
  }

and add var _ = require('underscore'); at the top
2. add after this if statement:

  } else if (_.isArray(magic.controllers.controllers)) {
    magic.controllers.controllers = _.map(magic.controllers.controllers, function (_path) {
        return path.resolve(appRoot, _path);
    })
  }

Please support this. I have 2 use cases for that and it will make my life a little bit easier :)

Thanks,
Boris.

The swagger validator middleware (from swagger-tools) added the 'options' param in order to set flag for validating responses
https://github.com/apigee-127/swagger-tools/blob/master/middleware/2.0/swagger-validator.js#L44

Support for that inside
https://github.com/apigee-127/magic/blob/master/lib/middleware.js
when using the magic module will be welcome :)

Thanks,
Boris

I've found that an empty array in default.yaml

someParameter: []
anotherParameter: 5

is being transferred to the Swagger file as a null:

x-a127-config:
  someParameter: &someParameter
  anotherParameter: &anotherParameter 5

This is due to lib/loader.js treating empty arrays as simply shorter verisons of non-empty arrays, when in fact Yaml requires them to be explicitly noted as empty using [].

This is causing errors for us as we often use empty arrays as defaults for array-type config values.

Not sure if the problem is in this code, or downstream in swagger-tools or json refs, but...

swagger-tools has been updated to version 0.9.14 which uses the latest version of json-refs - 2.1.5

This module specifies to use ^0.9.0 of swagger-tools so if you do a clean install it gets version 0.9.14 of swagger-tools and version 2.1.5 of json-refs

If you deploy a proxy to apigee edge with these versions, you get this errors when you try to call it

TypeError: Expected argument of type object, but instead had type object at clone (/organization/environment/api/node_modules/a127-magic/node_modules/swagger-tools/node_modules/json-refs/index.js:69) at /organization/environment/api/node_modules/a127-magic/node_modules/swagger-tools/node_modules/json-refs/index.js:70 at clone (/organization/environment/api/node_modules/a127-magic/node_modules/swagger-tools/node_modules/json-refs/index.js:69) at /organization/environment/api/node_modules/a127-magic/node_modules/swagger-tools/node_modules/json-refs/index.js:1165 at notifyIsolated (/organization/environment/api/node_modules/a127-magic/node_modules/swagger-tools/node_modules/json-refs/node_modules/native-promise-only/lib/npo.src.js:120) at notify (/organization/environment/api/node_modules/a127-magic/node_modules/swagger-tools/node_modules/json-refs/node_modules/native-promise-only/lib/npo.src.js:97) at drain (/organization/environment/api/node_modules/a127-magic/node_modules/swagger-tools/node_modules/json-refs/node_modules/native-promise-only/lib/npo.src.js:67) at processImmediate (timers.js:345)

Trying to use environment specific configurations on Edge. However, it only seems to pick up default.yaml on a test environment, and never test.yaml.

I've logged the environment variables, and APIGEE_ENVIRONMENT contains the correct value. However, magic is looking for A127_ENV or NODE_ENV when attempting to load the environment config, which don't appear to be defined.

Is it recommended to define either NODE_ENV or A127_ENV manually based on the APIGEE_ENVIRONMENT at the application level before calling init instead?

Authentication in one of our dev APIs was failing.

A client secret is defined as an integer within a config yaml file. Since the change over to js-yaml within this library, this means it's now being interpreted differently for some reason, and being rounded up.

Forcing it to a string by wrapping it within quotes does solve this problem but it's a major change in previous behaviour; which was quite unexpected and has caused a significant amount of time tracking this down on our side.

Example:

anAPI:
  clientSecret: 8421999999999995999

a127.init(function (config) {
    console.log(config.anAPI.clientSecret)
})

clientSecret becomes 8421999999999996000

Using 0.12.1 of a127-magic. Reverting to 0.11.2 this issue does not occur.

Scott,

File upload using formData, fails starting with a127-magic v0.11.1. Version v0.11.0 works fine.

Code snippets:

swagger.yaml

      consumes:
        - multipart/form-data

      parameters:
        - in: formData
          name: audioFile
          description: audioFile
          required: true
          type: file

debug output:

  "message": "Request validation failed: Parameter (audioFile) is required",
  "stack": "Error: Parameter (audioFile) is required\n    at throwErrorWithCode 
  .......
  ......

It seems as if the actual file was missing. When the parameter is required, the validator fails, when the parameter is set to optional then req.swagger.params.audioFile.value is undefined.

Any help would be greatly appreciated!

Thanks

This no longer works when deployed to Apigee edge because the runtime on there is v0.10 and one of the dependencies now installed is node-mime v2 which uses the const keyword

The dependency is introduced from the dependency to
https://jl.githost.io/apigee/swagger-tools which depends on
https://github.com/whitlockjc/json-refs (has to be version 2 - version 3 contains some new syntax which also wont work) which depends on
https://github.com/whitlockjc/path-loader which depends on superagent. If the version of superagent in here is specified as 3.6.0 then everything works

So for this to work, path-loader has to specify version 3.6.0 of superagent, json-refs version 2.* has to specify the correct version of path-loader, and swagger-tools has to specify the correct version of json-refs and this has to reference the correct version of swagger-tools

Hey All!

Trying to deploy a proxy to Apigee via 127, and I'm running into a strange issue. I'm getting the error below.

$ a127 project deploy
Deploying project a127-starter to dobson...
[Error: Error uploading resource /Users/ApigeeCorporation/node/a127-starter/node_modules/a127-magic: 413
{"fault":{"faultstring":"Body buffer overflow","detail":{"errorcode":"protocol.http.TooBigBody"}}}]

Any ideas on how I should size down the repo?

I'm using a127-magic, but wondering how to implement usual swagger-tools' security handlers instead of using x-a127-services?

Currently, my API returns:
{ "message": "unknown security handler: key", "code": "server_error", "statusCode": 403 }
to every call, which is good.

Please update to swagger-tools 0.9.0 which fixed this issue:
apigee-127/swagger-tools#234