wissance / ferrum Goto Github PK

We should provide very simple && lightweight UI without popular frameworks just with using pure ass css and vanila js

What should we have in our minimal UI:

1. Login / Logout to Keycloak as SuperUser and/or as Ordinary User
2. Create Realm, Client, Users for SuperUser via UI
3. Update/Reset password of Users (SuperUser - all, OrdinaryUser - own)

Check how would behave Ferrum if we stop Redis server, it shouldn't crashed it should respond with something like "Data Storage is not ready, please try again later", and work when storage is Up

In this task i also take some additional works:

• add typical error for DataStorageNotAvailable
• add more typical errors 4 determine what happened (ObjectNotFoundError and others), this point solves issue: #22
• working with typical errors (previous list item) brake me a lot of manager methods: Have to implement minimal set of tests (partially solves issue #62
• I should also make come clean-up in Dockerfile && docker-compose : add permanent name for containers, volumes and so on
• working on docker compose i change to init redis via sh. script (not using python) #44
• test all method on availability

now we expect must to have it in our JSON, and strategy is only to check password equality we should to use it in our logic

• Add most significant functions and types documentation for package data
• Add most significant functions and types documentation for package managers
• Add most significant functions and types documentation for package services
• Add most significant functions and types documentation for package application
• Add most significant functions and types documentation for package api
• Add Readme changes about usage with Redis

Currently we store quite a big data (UserSession) in a server memory, for 10K online users we could consume a lot of memory (5K10K users=50M bytes, 5K100K users=500M), probably? we should move it in a DataStorage (for Redis yes, for files no).

For Redis this keys must have TTL and must be destroyed by Redis itself, for 100K users 500M is OK, but in future we should consider this issue

We should encode userdata for preventing it from access via getting access to data storage (Redis, File). Probably each user credentials should be stored as Base-64 encoded string, this feature should be taken into account in Ferrum CLI too

Wissance will provide an enterprise biometry server, we should integrate it in authorization process

to perform all CLI operations we should use credentials to authorize and run these operations. We should create admin account via CLI and save it in DataStorage and save it. For FILE data provider admin account must be present in data.json.

after expiration we expect to have 200 status when calling Introspect:

	// 5. Expire token by timeout and got 401 (Unauthorized) status
	time.Sleep(time.Second * time.Duration(testAccessTokenExpiration))
	userInfo = getUserInfo(t, baseUrl, realm, token.AccessToken, "401 Unauthorized")
	// todo(UMV): this one looking strange because token expired and we expect here 200 as status
	tokenIntResult = checkIntrospectToken(t, baseUrl, realm, token.AccessToken, testClient1, testClient1Secret, "200 OK")

Ensure this is right behaviour and fix if not

for other services like AD, ldap, FreeIPA

Introspect uses pair client_id:client_secret pair decoded in base-64 for token Introspection, check how this endpoint works with public client and fix if we have behaviour mismatch

[authorization code mechanism ] (https://www.keycloak.org/docs/latest/securing_apps/#_oidc)

This task is not about full authorization code flow because it depends on User Agent, but we here develop backend only, however, this part is up to Ferrum backend:

The application then uses the authorization code along with its credentials to obtain an Access Token, Refresh Token and ID Token from Keycloak.

In this task we should create:

1. Docker compose as conmponent orchestrator
2. Add Redis image with configurable parameters and persistance storage:
   • user
   • password
   • host+port (address)
   • db number
3. Add run application with settings override:
   • network settings (host+port), if schema is HTTPS we should generate certs pair
   • redis settings (namespace && connect options)
4. Update documentation about how to use

After we added support 4 running Ferrum on Redis we must ensure that it works as we expected,

1. At least 1M users in Redis and 10K online, we expect 100ms and less for every request

I.e. Keycloak Token introspect contains following info:

{
    "exp": 1601824811,
    "iat": 1601824511,
    "jti": "d5a4831d-7236-4686-a17b-784cd8b5805d",
    "iss": "http://localhost:8083/auth/realms/baeldung",
    "sub": "a5461470-33eb-4b2d-82d4-b0484e96ad7f",
    "typ": "Bearer",
    "azp": "jwtClient",
    "session_state": "96030af2-1e48-4243-ba0b-dd4980c6e8fd",
    "preferred_username": "[email protected]",
    "email_verified": false,
    "acr": "1",
    "scope": "profile email read",
    "DOB": "1984-07-01",
    "organization": "baeldung",
    "client_id": "jwtClient",
    "username": "[email protected]",
    "active": true
}

our contains only active

Unfortunately we hava forgot to add it

Tests are running like this:

func TestApplicationOnHttp(t *testing.T) {
	testRunCommonTestCycleImpl(t, &httpAppConfig, "http://127.0.0.1:8284")
}

// todo(UMV): take from config
func TestApplicationOnHttps(t *testing.T) {
	http.DefaultTransport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
	testRunCommonTestCycleImpl(t, &httpsAppConfig, "https://127.0.0.1:8672")
}

But we have settings in variables, we should use setting value to build URL and run

We should add reading data not only from file but also from MongoDB

Basic functionality for beta version:

• implementation of 2 methods to get new token and to check userInfo
• control user session by timeouts (timeout must set for realm, or used default values)
• design API that is compatible with KeyCloak (we could use Wissance.Authorization project to test)
• ensure that generated JWT contains valid information
• use json as a storage, add app create method with passing Json's not path to data && server file (parallel methods)
• add test that includes full cycle (integrational tests)
• check how it works with nerzal/gocloak

According to Authrorization Flow we should implement keycloak like Authorization Endpoint

~/auth/realms/{{realm}}/protocol/openid-connect/auth?response_type=code&client_id=jwtClient

Moreover, this endpoint accepts scope and redirect_uri as optional parameters.

We won’t use this endpoint in Postman. Instead, we usually initiate the authorization code flow via a browser. Then Ferrum redirects the user to a login page if no active login cookie is available. Finally, the authorization code is delivered to the redirect URL.

It is possible to pass some non valid data in Realm i.e. pass sql injection (now we don't support SQL databases but could do in a future) or something that could harm a server, therefore we must validate text values. Therefore we should add /api/rest/utilities.go where we should add func Validate(strVal string) bool, err. If param is inavlid we should return 400 error with message like {ParamName} is Invalid .

Realm probably could contains only:

• letters,
• digits,
• _ (underscore)
• • (dash)

However one of SQL injection technique is to pass sql commentary --, therefore we must be sure that Realm parameter is safe for our system

When logging configuretion not provided with FileType appender, we got panic here:

// Setting up listener for logging
  appenderIndex := app.logger.GetAppenderIndex(config.RollingFile, app.appConfig.Logging.Appenders)
  app.httpHandler = app.createHttpLoggingHandler(appenderIndex, router)
  return nil

Currently we are checking only statuses

after all works on Redis as provider will be finished we have to make admin interface for Ferrum configure

Admin interface must provide following common operations schema: {operation} {resource} {resource_id} {params} {value}

1. {operation} one of the 4 operation create, update, delete, get
2. {resource} - name of resource - realm, client, user
3. {namespace} - object namespace
4. {resource_id} - resource name i.e. name of realm, client, user
5. {params} additional params that requires for operations i.e. realm for client and user
6. {value} is a JSON that contains object value itself

i.e. consider Realm creation:

   ferrum-admin --operation=create --resource=realm --namespace=ferrum_2 --resource_id=IRP --value=\"{\"name\" "irp_rt\", ....}\"

If we provide full realm structure with client and user this data struct should be separated && created all objects at once

if we create a User we should operate with public user information (not show credentials), if we run user creation we should query user password && check password

additionally we have 2 additional methods

1. ferrum-admin --operation=change_password --namespace=ferrum_2 --resource=user --resource_id=admin
2. ferrum-admin --operation=reset_password --namespace=ferrum_2 --resource=user --resource_id=admin

reset_password resets password to randomly generated (8 symbols at least)

Techically ferrum-admin should be additional software located in /ferrum/api/admin/cli, management operations (except password operations) must be declared in DataContext interface

we should add logging to see what errors, warnings and so on

• add trace logs to see what happend on negative cases
• add debug to see more detailed info
• add HTTP logging to see result of api query

• Yandex,
• Mail,
• gosuslugi

1. To run locally without Docker
2. To run in DOCKER with env Vars substitution

It should work like works token refresh in KeyCloak

in KeyCloak we are sending HTTP-request to token endpoint auth/realms/master/protocol/openid-connect/token
but instead of passing username && password we are passing refresh_token through the body of a request (x-www-form-urlencoded) simulteneously with refresh_token value for key grant_type

Implement following analysis:

• SAST
• DAST

We should change data init from python script to our own CLI,
we should also use approach and data like in \Ferrum\tools\create_wissance_demo_users.ps1
with SUCH values like WissanceFerrumDemo as Realm name, except user should not be Michael Ushakov,
we have to set 2 users instead of umv:

1. John Doe (jodo as preffered_username)
2. Jane Doe (jado as preffered_username)

1. Create Realm without client just realm
2. Get realm via admin interface
3. Error getting realm

we should check and fix

write readme with description what this project is about and how to use it

Currently we used hardcoded value instead of configuration

we pass scope for token generation. We should also think how to use properly profile and e-mail scopes

when starting Ferrum as application we initialize it as follows:

    app := application.CreateAppWithConfigs("./config.json", "./data.md", "./keyfile")

We should be able to pass only name of the first main config (otherwise use default name - ./config.json)

Implement 2 factor auth via:

• e-mail code
• mobile phone (but don't know how to do this)

We should think about Data Scope i.e. within Realm there could be different Scopes, each Scope is like an isolated mini Realm, this is actual 4 User Multitenancy apps

• Swagger must be display only in development mode (this mode also should be passed to Docker)
• Additionally what is more important is to add static page with endpoints like in keycloak (relative part will be added later)
  auth/realms/ubsdev/.well-known/openid-configuration

We should create data generator for performance tests, probably we should generate test data via python we should have at least 10 realms with 3-5 clients in them and 10-50k users

We should provide known users identifiers based on pattern i.e

• realm1, ....realm10
• client1_1, ......client5_1 , ...client1_10 , ....client5_10
• user1_1, .....user20000_1, user1_10, ....user50000_10

After we have this perf test data we could check performance in comparison with KeyCloak (probably we also should have Keycloak data generator too

We should make following docker improvements:

• Config for Ferrum should form with env variables, it shouldn't be hardcoded (i.e. Redis connect options)
• We should add variable in env that is responsible for running app in devmode or not (-devmode)
• We should also configure log level via env
• Redis user && password and save (on disk) option also should came from env (at present moment we pass them directly in docker-compose )

In this tests we should either check result and returning error:

• Realm methods tests
• Client methods tests
• Users methods tests

Optionally:

• Test FileDataManager methods

Minimal app must be:

• behave like keycloak auth server (get token, validate users)
• manageable, so we must have possibility to start and stop app
• allow to create custom data (realms, clients, users)

In new version /auth/ was removed from path, therefore we should add proxy routes
Newer version of Keycloak should be supported parallel to elder ones. This Feature MUST be implemented by adding new routes with URLs without auth handling by the same Handlers.