Giter Club home page Giter Club logo

commitstream.web's People

Contributors

amitair avatar boraaksoy avatar caspergrl avatar corywheeler avatar danielgruesso avatar darchitetto avatar dsaldric avatar gitter-badger avatar iradennis avatar jallers avatar jogoshugh avatar kunzimariano avatar matiasheffel avatar mhh avatar mirvin avatar pablofreiberg avatar shawnmarie-zz avatar superman32432432 avatar tadeboye avatar veplechuc avatar walkerrandolphsmith avatar

Stargazers

avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

kunzimariano jogoshugh matiasheffel deveo pjboudrx kummitus

commitstream.web's Issues

Discovery: Support TFS TFVC tfvc.checkin payload

Background

We are researching the tfvc.checkin payload to determine whether we can write a translator on our CommitStream service to get enough information to render several links to the TFS UI for users to follow from a panel in our own application.

Our current git.push support

The following can be viewed on one of our public testing instances at VersionOne Development api-examples instance. You will need to use admin / admin for credentials.

image

Here’s a break down of what we currently produce for the git.push event:

There are three important links that we are able to extract out of the git.push payload.

  1. Commit / changeset URL: c501cb: links to the actual commit in the UI -> https://v1platformtest.visualstudio.com/_git/V1%20Integration/commit/c501cb4aa7dc118576d09bd137eaa7be50181a62
  2. Branch URL: MyNew/Shelveset: links to the branch in the UI ->
    https://v1platformtest.visualstudio.com/_git/V1%20Integration/#version=GBMyNew%2FShelveset
  3. Project / Repository URL: v1platformtest/V1 Integration: links to the root of the code base in the UI ->
    https://v1platformtest.visualstudio.com/_git/V1%20Integration

We do not currently link the author name to anything.

Code for parsing git.push payloads for these URLs

Our code is here: https://github.com/openAgile/CommitStream.Web/blob/S-60961_TFS_OnPremise/src/app/api/translators/es6/vsoGitTranslator.js#L66-L93

 getProperties(event) {
    const commit = event.commit;
    const branch = event.branch;
    const html_url = event.html_url;
    const props = {
      repo: '',
      branchHref: '',
      repoHref: ''
    };

    const urlComponents = html_url.match(/http.?:\/\/..*?_git\/(.*?)\/commit/);
    const serverUrlMatch = html_url.match(/(http.?:)\/\/(.*?_git)\//);

    if (urlComponents !== null && serverUrlMatch !== null) {
      const repoName = urlComponents[2];
      const repoOwner = urlComponents[1];
      const protocol = serverUrlMatch[1];
      const serverUrl = protocol + '//' + serverUrlMatch[2];
      props.repo = `${repoOwner}/${decodeURIComponent(repoName)}`;
      props.repoHref =  serverUrl + '/' + repoName;
      props.branchHref =  props.repoHref + '/#version=GB' + encodeURIComponent(branch);
    } else {
      // TODO: use proper error here
      throw 'Could not parse VsoGitCommitReceived event props correctly.';
    }

    return props;
  }

Research on tfs.checkin payload and public documentation differences

We are confused by the documentation on the web site and what we are actually seeing from VSTS in our cloud instances.

Documented tfvc.checkin payload format for resourceVersion 1.0 is not being sent from VSTS

From this document: https://www.visualstudio.com/en-us/docs/marketplace/integrate/service-hooks/events

We see that the resourceVersion = 1.0 payload should be like this, noting that we have the resourceContainers.project.id = be9b3917-87e6-42a4-a549-2bc06a7a878f

{
  "id": "f9b4c23e-88dd-4516-b04d-849787304e32",
  "eventType": "tfvc.checkin",
  "publisherId": "tfs",
  "scope": "all",
  "message": {
    "text": "Normal Paulk checked in changeset 18: Dropping in new Java sample",
    "html": "Normal Paulk checked in changeset <a href=\"https://fabrikam-fiber-inc.visualstudio.com/web/cs.aspx?pcguid=d81542e4-cdfa-4333-b082-1ae2d6c3ad16&amp;cs=18\">18</a>: Dropping in new Java sample",
    "markdown": "Normal Paulk checked in changeset [18](https://fabrikam-fiber-inc.visualstudio.com/web/cs.aspx?pcguid=d81542e4-cdfa-4333-b082-1ae2d6c3ad16&cs=18): Dropping in new Java sample"
  },
  "detailedMessage": {
    "text": "Normal Paulk checked in changeset 18: Dropping in new Java sample",
    "html": "Normal Paulk checked in changeset <a href=\"https://fabrikam-fiber-inc.visualstudio.com/web/cs.aspx?pcguid=d81542e4-cdfa-4333-b082-1ae2d6c3ad16&amp;cs=18\">18</a>: Dropping in new Java sample",
    "markdown": "Normal Paulk checked in changeset [18](https://fabrikam-fiber-inc.visualstudio.com/web/cs.aspx?pcguid=d81542e4-cdfa-4333-b082-1ae2d6c3ad16&cs=18): Dropping in new Java sample"
  },
  "resource": {
    "changesetId": 18,
    "url": "https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_apis/tfvc/changesets/18",
    "author": {
      "id": "d6245f20-2af8-44f4-9451-8107cb2767db",
      "displayName": "Normal Paulk",
      "uniqueName": "[email protected]"
    },
    "checkedInBy": {
      "id": "d6245f20-2af8-44f4-9451-8107cb2767db",
      "displayName": "Normal Paulk",
      "uniqueName": "[email protected]"
    },
    "createdDate": "2014-05-12T22:41:16Z",
    "comment": "Dropping in new Java sample"
  },
  "resourceVersion": "1.0",
  "resourceContainers": {
    "collection": {
      "id": "c12d0eb8-e382-443b-9f9c-c52cba5014c2"
    },
    "account": {
      "id": "f844ec47-a9db-4511-8281-8b63f4eaf94e"
    },
    "project": {
      "id": "be9b3917-87e6-42a4-a549-2bc06a7a878f"
    }
  },
  "createdDate": "2016-09-19T13:03:26.2056408Z"
}

Actual VSTS tfvc.checkin payload for resourceVersion 1.0

However, when we set issue actual hook payloads from our repo, we don’t see this, instead we see this:

{
  "subscriptionId": "20019a9b-3534-4705-a755-198d23de2ed7",
  "notificationId": 6,
  "id": "b72be65b-614d-4652-9ca9-11d2942a5c91",
  "eventType": "tfvc.checkin",
  "publisherId": "tfs",
  "scope": "all",
  "message": {
    "text": "Josh Gough checked in changeset 17: Updated README.md S-12345",
    "html": "Josh Gough checked in changeset <a href=\"https://v1platformtest.visualstudio.com/web/cs.aspx?pcguid=6b1d2d92-0bb2-4ff1-b1e0-5b79fd2abd67&amp;cs=17\">17</a>: Updated README.md S-12345",
    "markdown": "Josh Gough checked in changeset [17](https://v1platformtest.visualstudio.com/web/cs.aspx?pcguid=6b1d2d92-0bb2-4ff1-b1e0-5b79fd2abd67&cs=17): Updated README.md S-12345"
  },
  "detailedMessage": {
    "text": "Josh Gough checked in changeset 17: Updated README.md S-12345",
    "html": "Josh Gough checked in changeset <a href=\"https://v1platformtest.visualstudio.com/web/cs.aspx?pcguid=6b1d2d92-0bb2-4ff1-b1e0-5b79fd2abd67&amp;cs=17\">17</a>: Updated README.md S-12345",
    "markdown": "Josh Gough checked in changeset [17](https://v1platformtest.visualstudio.com/web/cs.aspx?pcguid=6b1d2d92-0bb2-4ff1-b1e0-5b79fd2abd67&cs=17): Updated README.md S-12345"
  },
  "resource": {
    "hasMoreChanges": true,
    "teamProjectIds": [
      "b70385b4-ae0f-4afd-b166-6aff62bfd0b0"
    ],
    "changesetId": 17,
    "url": "https://v1platformtest.visualstudio.com/_apis/tfvc/changesets/17",
    "author": {
      "id": "0b88cae0-021f-4fa0-b723-d670c74ae474",
      "displayName": "Josh Gough",
      "uniqueName": "[email protected]",
      "url": "https://app.vssps.visualstudio.com/A95780de7-c7d8-4742-a6e7-12e311437415/_apis/Identities/0b88cae0-021f-4fa0-b723-d670c74ae474",
      "imageUrl": "https://v1platformtest.visualstudio.com/_api/_common/identityImage?id=0b88cae0-021f-4fa0-b723-d670c74ae474"
    },
    "checkedInBy": {
      "id": "0b88cae0-021f-4fa0-b723-d670c74ae474",
      "displayName": "Josh Gough",
      "uniqueName": "[email protected]",
      "url": "https://app.vssps.visualstudio.com/A95780de7-c7d8-4742-a6e7-12e311437415/_apis/Identities/0b88cae0-021f-4fa0-b723-d670c74ae474",
      "imageUrl": "https://v1platformtest.visualstudio.com/_api/_common/identityImage?id=0b88cae0-021f-4fa0-b723-d670c74ae474"
    },
    "createdDate": "2017-01-20T16:28:45Z",
    "comment": "Updated README.md S-12345"
  },
  "resourceVersion": "1.0",
  "resourceContainers": {
    "collection": {
      "id": "6b1d2d92-0bb2-4ff1-b1e0-5b79fd2abd67",
      "baseUrl": "https://v1platformtest.visualstudio.com/"
    },
    "account": {
      "id": "95780de7-c7d8-4742-a6e7-12e311437415",
      "baseUrl": "https://v1platformtest.visualstudio.com/"
    }
  },
  "createdDate": "2017-01-20T16:28:47.755774Z"
}

The resourceVersion 1.0-preview.1 does not match either

We also tried to change to the resourceVersion 1.0-preview.1 and that also is different:

{
  "subscriptionId": "20019a9b-3534-4705-a755-198d23de2ed7",
  "notificationId": 8,
  "id": "9281bf99-f899-4de5-81ee-6ee990259d90",
  "eventType": "tfvc.checkin",
  "publisherId": "tfs",
  "scope": "all",
  "message": {
    "text": "Josh Gough checked in changeset 19: Updated README.md S-12345",
    "html": "Josh Gough checked in changeset <a href=\"https://v1platformtest.visualstudio.com/web/cs.aspx?pcguid=6b1d2d92-0bb2-4ff1-b1e0-5b79fd2abd67&amp;cs=19\">19</a>: Updated README.md S-12345",
    "markdown": "Josh Gough checked in changeset [19](https://v1platformtest.visualstudio.com/web/cs.aspx?pcguid=6b1d2d92-0bb2-4ff1-b1e0-5b79fd2abd67&cs=19): Updated README.md S-12345"
  },
  "detailedMessage": {
    "text": "Josh Gough checked in changeset 19: Updated README.md S-12345",
    "html": "Josh Gough checked in changeset <a href=\"https://v1platformtest.visualstudio.com/web/cs.aspx?pcguid=6b1d2d92-0bb2-4ff1-b1e0-5b79fd2abd67&amp;cs=19\">19</a>: Updated README.md S-12345",
    "markdown": "Josh Gough checked in changeset [19](https://v1platformtest.visualstudio.com/web/cs.aspx?pcguid=6b1d2d92-0bb2-4ff1-b1e0-5b79fd2abd67&cs=19): Updated README.md S-12345"
  },
  "resource": {
    "hasMoreChanges": true,
    "teamProjectIds": [
      "b70385b4-ae0f-4afd-b166-6aff62bfd0b0"
    ],
    "changesetId": 19,
    "url": "https://v1platformtest.visualstudio.com/_apis/tfvc/changesets/19",
    "author": {
      "id": "0b88cae0-021f-4fa0-b723-d670c74ae474",
      "displayName": "Josh Gough",
      "uniqueName": "[email protected]",
      "url": "https://app.vssps.visualstudio.com/A95780de7-c7d8-4742-a6e7-12e311437415/_apis/Identities/0b88cae0-021f-4fa0-b723-d670c74ae474",
      "imageUrl": "https://v1platformtest.visualstudio.com/_api/_common/identityImage?id=0b88cae0-021f-4fa0-b723-d670c74ae474"
    },
    "checkedInBy": {
      "id": "0b88cae0-021f-4fa0-b723-d670c74ae474",
      "displayName": "Josh Gough",
      "uniqueName": "[email protected]",
      "url": "https://app.vssps.visualstudio.com/A95780de7-c7d8-4742-a6e7-12e311437415/_apis/Identities/0b88cae0-021f-4fa0-b723-d670c74ae474",
      "imageUrl": "https://v1platformtest.visualstudio.com/_api/_common/identityImage?id=0b88cae0-021f-4fa0-b723-d670c74ae474"
    },
    "createdDate": "2017-01-20T16:35:01Z",
    "comment": "Updated README.md S-12345"
  },
  "resourceVersion": "1.0-preview.1",
  "createdDate": "2017-01-20T16:35:03.4905568Z"
}

Current attempts to parse URLs from tfvc.checkin payloads for resourceVersion 1.0 that we receive from VSTS

From the currently sent 1.0, we are able to produce a few URL through implicit knowledge that we discovered by hacking around on the address bar in our browser.

We would prefer to get these URLs in the payload itself, similar to what we do in our own API by production HAL resources that have a special _links: {} object that provides links to relevant resources. You can see an example in our test case here.

But, if we can know that this 1.0, or what is documented on the web site, will be the actual, stable payload for some time to come, we will feel better about coding this knowledge into our translator.

  1. Commit / changeset URL: ->
    payload.resourceContainers.collection.baseUrl + “/” + payload.resource.teamProjectIds[0] + . “/_versionControl/changeset/” + payload.resource.changesetId
  2. Branch URL: --> Unknown. There is some info here about branching in TFVC, Branching in TFVC, but I have not been able to figure out how to create branches in TFVC, nor whether the payload would contain this information for us.
  3. Project / Repository URL: -> payload.resourceContainers.collection.baseUrl + “/” + payload.resource.teamProjectIds[0] + . “/_versionControl/

10/10 Notes

​​Issue​s / Enhancements:

  • ​Most recent events showing up at top of list: will we have to resort to a different Read Model data store or projection or ES query to support ordering an api query by commit date?

​​Tech Debt

  • Error handling and unit testing of V1 core side.
    • How can we prevent Core builds from being impacted by a remote service call on App Start?
  • ​​Unit testing of import functionality and translation
    • Match translator by specific GitHub message version preferentially
    • Fallback to trying to peel out the required json elements from message
    • ​​Error handling in node side
  • Logging to sumo? Estimably lessons?
  • Notifications?

​​API response format

  • common API formats like HAL and CollectionJson, etc
  • paging information in response
  • parameters for page size
  • Docs from embedded code comments
    • What are the best tools for this?

Installation docs review

  • Talk to product specialists
  • How much can we automate away entirely?
    • YAML file for EventStore can be powershell sc produced
  • Have more people try them
  • Can we describe how to develop under Linux or Mac for people who want to run event store with Mono?

New ​​Features

  • ​Registering a repo event hook via an API call
    ​* ​In-V1 experience for configuring GitHub repos, similar to Azure experience: starts to get to Digest concept
    ​* ​Paging / infinite scroll of previous commits in asset detail

Update partition-commits-for-instance-by-versionone-workitem

Looks like one of the hotfixes did not get applied into the code.

We have this in production:

var getWorkitems = function(message) {
  var re = new RegExp("[A-Z,a-z]{1,2}-[0-9]+", "g");
  var matches = message.match(re);
  return matches;
}

var callback = function(state, ev) {
  if (ev && ev.eventType && ev.eventType[0] != '$' && ev.metadata && ev.metadata.instanceId 
      && ev.data && ev.data.commit && ev.data.commit.message) {
    var workItems = getWorkitems(ev.data.commit.message);
    var withOutDuplicates = workItems.filter(function (i, p) {
        return workItems.indexOf(i) == p;
    });
    withOutDuplicates.forEach(function(workItem) {
      workItem = workItem.toUpperCase();
      linkTo('versionOne_CommitsWithWorkitems-' + ev.metadata.instanceId + '_' + workItem, ev);
    });
  }
};

fromCategory('versionOne_CommitsWithWorkitemMention')
  .whenAny(callback);

But, it's different in the code. Need to review and resolve.

Must specify ApiKey as a querystring parameter

Hi,
I am trying to run commitstream locally. When I hit 127.0.0.1:6565 it asks for an api key. When I specify the api key i gave commitstream in the form 127.0.0.1:6565/?apikey= it does not work giving me same error.

Which api key is it looking for?

Azure internally uses http even when initial request is HTTPS

Because Azure uses http behind it's websites facade process, even though we make an initial request at https://, the Node process is internally running on HTTP, so this code returns http:// instead of https://:

app.get('/app', function(req, res) {
    res.setHeader('content-type', 'application/javascript');
    var protocol = req.protocol;
    var host = req.get('host');

    res.render('app', {
        apiUrl: protocol + '://' + host + '/api/query?workitem=',
        templateUrl: protocol + '://' + host + '/assetDetailCommits.html'
    });
});

Remove Repository

As an API user, I must be able to remove an inbox from a digest.

Acceptance Criteria:

  1. User must be able to specify the instanceId and inboxId from which the inbox in question is to be removed
  2. User must be able to receive confirmation when the inbox is successfully removed
  3. User must be able to receive an error message when the inbox fails to be removed from the instance
  4. GET requests to the deleted inbox must return 410 (gone)
  5. POST requests to the deleted inbox must return 410 (gone)
  6. Historical commit data for the inbox will remain

Route
DELETE /api/:instanceId/inboxes/:inboxId?apiKey=:apiKey

Header
X-HTTP-Method-Override:delete

Inputs

  • instanceId
  • inboxId

Output

  • 200 Status Code
  • Message
  • instanceId
  • inboxId

https://github.com/expressjs/method-override
http://docs.geteventstore.com/http-api/3.0.1/deleting-a-stream//
http://stackoverflow.com/questions/2342579/http-status-code-for-update-and-delete

GET requests to the deleted inbox must return 410 (gone) AT-35167

Historical commit data for the inbox will remain AT-35169

POST requests to the deleted inbox must return 410 (gone) AT-35168

User must be able to receive an error message when the inbox fails to be removed from the instance AT-35166

User must be able to receive confirmation when the inbox is successfully removed AT-35165

User must be able to specify the instanceId and inboxId from which the inbox in question is to be removed AT-35164

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.