Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Policy Factory (Alpha Release) #2855

Merged
merged 2 commits into from
Aug 15, 2023
Merged

Policy Factory (Alpha Release) #2855

merged 2 commits into from
Aug 15, 2023

Conversation

jvanderhoof
Copy link
Contributor

@jvanderhoof jvanderhoof commented Jul 11, 2023

This work is an extension of the my Hackathon effort from this past fall, which aimed enable an API for generating a variety of Conjur resources.

This effort:

  • Moves policy rendering into Conjur, which allows the Factory templates to be more tightly managed
  • Uses JSON Schema to define required and optional parameters, and validates incoming requests
  • Adds a tool for validating Policy template rendering, to aid in development

Overview

Factory Structure

Factories are stored in Conjur Variables as Base64 encoded values. This PR includes the following factories:

  • core
    • user
    • group
    • policy
    • managed-policy
    • grant
  • authenticators
    • authn-oidc
  • connections
    • database

Factory variables follow the following convention:

conjur/factories/<classification>/<factory>

The following factories will be added as part of a separate pull request:

  • core
    • host
    • webservice
    • variable
    • permit
    • deny
    • revoke
  • authenticators
    • authn-jwt
    • authn-azure
    • authn-gcp
    • authn-k8s
    • authn-ldap
  • connections
    • API - using a key

Tests for these templates will also be part of this upcoming PR.

API

The API selects the desired policy (ex. core or authenticators) as well as the target variable name (group, policy, user, managed-policy, etc), we have an immense amount of flexibility to organize and create factories in the future.

Using JSON Schema, each factory defines the inputs it requires and and optionally accepts. This information is available through the factory's info endpoint. This allows us to dynamically include these endpoints in CLIs and SDKs in the future.

Security

Factory endpoint requests require execute permission on the Factory variable and appropriate permission on the target Policy. This allows Conjur RBAC to be used to manage who/what can use Factories to extend policy, and all actions are captured in Conjur Audit.

Demo

To install the base policies required for the Policy Factory, run the following command on Conjur (Conjur needs to be running):

CONJUR_AUTHN_API_KEY=<admin-api-key> rake policy_factory:load

The above command will install a base policy into the conjur/factories namespace. This base policy includes factories for creating Conjur (all in the core namespace):

  • Group
  • ManagedPolicy - creates a policy with an owner group
  • Policy
  • User

With the above factory templates, the following API endpoints become available:

API Overview

The API is fully documented in the Solution Design.

Connected Issue/Story

N/A

Definition of Done

At least 1 todo must be completed in the sections below for the PR to be
merged.

Changelog

  • The CHANGELOG has been updated, or
  • This PR does not include user-facing changes and doesn't require a
    CHANGELOG update

Test coverage

  • This PR includes new unit and integration tests to go with the code
    changes, or
  • The changes in this PR do not require tests

Documentation

  • Docs (e.g. READMEs) were updated in this PR
  • A follow-up issue to update official docs has been filed here: [insert issue ID]
  • This PR does not require updating any documentation

Behavior

  • This PR changes product behavior and has been reviewed by a PO, or
  • These changes are part of a larger initiative that will be reviewed later, or
  • No behavior was changed with this PR

Security

  • Security architect has reviewed the changes in this PR,
  • These changes are part of a larger initiative with a separate security review, or
  • There are no security aspects to these changes

)
end
rescue => e
return @failure.new(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid too many return statements within this method.


require 'base64'

module Factories
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar blocks of code found in 2 locations. Consider refactoring.


require 'base64'

module Factories
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar blocks of code found in 2 locations. Consider refactoring.

policy: Base64.encode64(policy_template),
target_policy_namespace: "<%= branch %>",
target_variable_namespace: "<%= branch %>/<%= id %>",
schema: {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Identical blocks of code found in 2 locations. Consider refactoring.

version: 1,
policy: Base64.encode64(policy_template),
policy_branch: "<%= branch %>",
schema: {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Identical blocks of code found in 2 locations. Consider refactoring.

@jvanderhoof jvanderhoof force-pushed the policy-factory-v2 branch 2 times, most recently from e20cd4b to 3ca23b5 Compare July 14, 2023 21:11
@failure = ::FailureResponse
end

def call(factory_template:, request_body:, account:, authorization:)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method call has a Cognitive Complexity of 10 (exceeds 5 allowed). Consider refactoring.

@failure = ::FailureResponse
end

def call(factory_template:, request_body:, account:, authorization:)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method call has 42 lines of code (exceeds 25 allowed). Consider refactoring.


private

def validate_and_transform_request(schema:, params:)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method validate_and_transform_request has 26 lines of code (exceeds 25 allowed). Consider refactoring.

@failure.new(errors.flatten, status: :bad_request)
end

def render_and_apply_policy(policy_load_path:, policy_template:, variables:, account:, authorization:)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method render_and_apply_policy has 41 lines of code (exceeds 25 allowed). Consider refactoring.

end
end

def set_factory_variables(schema_variables:, factory_variables:, variable_path:, authorization:, account:)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method set_factory_variables has 27 lines of code (exceeds 25 allowed). Consider refactoring.

@jvanderhoof jvanderhoof force-pushed the policy-factory-v2 branch 2 times, most recently from 3b64fd2 to 4d23821 Compare July 19, 2023 13:21
@jvanderhoof jvanderhoof marked this pull request as ready for review July 19, 2023 13:23
@jvanderhoof jvanderhoof requested a review from a team as a code owner July 19, 2023 13:23
Copy link
Contributor

@john-odonnell john-odonnell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't reviewed the templates or tests yet, but this looks great so far. I'll finish up my review tomorrow 🚀

Comment on lines 71 to 76
@resource.where(
Sequel.like(
:resource_id,
"#{account}:variable:conjur/factories/#{kind}/%"
)
).all.select { |i| i.resource_id.split('/').last == id }.max { |a, b| a.id <=> b.id }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's my understanding of this logic:

  1. @resource.where(...).all will return all factory variables of the specified kind:

    [core/v1/user, core/v2/user, core/v1/group]

  2. .select {...} reduces the results to only those of the specified ID:

    [core/v1/user, core/v2/user]

  3. .max {...} reduces this further to only the resource with the highest version:

    core/v2/user

I'm not certain of the mechanism that the <=> operator uses, but it seems to leave open some irregularity - for example, "v9" <=> "v10" returns 1, indicating "v9" is greater.

Definitely won't have an effect now, and probably not for a while (if ever), but thought I'd capture it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch John. I'll add some tests for this.

end
.map do |_, versions|
# find the most recent version
versions.max { |a, b| a.id <=> b.id }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as comment on line 76.

Comment on lines +115 to +126
@failure.new(
{ resource: "#{classification}/#{version}/#{id}", message: 'Requested Policy Factory is not available' },
status: :bad_request
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this failure case truly a :bad_request? Doesn't that imply client error, where this would be a server error (the variable exists, but doesn't contain the information we expect)?

Copy link
Contributor Author

@jvanderhoof jvanderhoof Aug 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about 404 Not Found instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is rough because it's dependent on prior actions (as you've pointed out). Micah and I have been talking about phase two of these factories, which makes them first class citizens at the policy level (work here). Much of the pre-population issues will go away with this work.

As this is only an alpha level feature, how do you feel about addressing this work as part of the next phase?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me 👍


require 'base64'

module Factory
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR has both a Factory and Factories module, but they aren't distinct.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I'll standardize on Factories.

Comment on lines +12 to +16
{
code: @response_codes[@response.status]
}.tap do |rtn|
rtn[:error] = format_error_message(@response.message)
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these two code samples have the same result, as .tap returns the object it's called on with modifications?

{
  code: @response_codes[@response.status]
}.tap do |rtn|
  rtn[:error] = format_error_message(@response.message)
end
{
  code: @response_codes[@response.status],
  error: format_error_message(@response.message)
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. The tap function is typically used if you need to optionally perform an action on something.


Setup will follow the following workflow:

```plantuml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't look like these are rendering correctly in the final MD.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. VSCode does it for me. I'll export the diagrams and include the images.

Comment on lines +23 to +28
- !variable provider-uri
- !variable client-id
- !variable client-secret
- !variable redirect-uri
- !variable claim-mapping
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we include the optional token-ttl, name & provider-scope?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've intentionally left these out as we need a way to create a variable only if the variable has a value. I'm not sure how best to do that yet.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make more sense to create the variables and leave them empty? Let default behavior ride, and let the user change config if needed.

@failure = ::FailureResponse
end

def call(factory_template:, request_body:, account:, authorization:)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only file where I feel the use of .bind handling the [Success|Failure]Responses has hindered readability & upped complexity.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my least favorite file in the commit. I'll take another look. I'm planning to refactor this in the future, and if it's gnarly, I'll file a ticket and give a comment here.

- !policy
id: <%= id %>
annotations:
factory: connections/database
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
factory: connections/database
factory: connections/v1/database

Comment on lines 29 to 50
"properties": {
"branch": {
"description": "Policy branch to load this group into",
"type": "string"
},
"annotations": {
"description": "Additional annotations to add to the group",
"type": "object"
}
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These properties refer to a group, and also seem incomplete - need [member|role]_resource_[id|type]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This schema is still missing [member|role]_resource_[id|type] properties


require 'spec_helper'

describe SuccessResponse do
Copy link
Contributor

@john-odonnell john-odonnell Jul 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the unit tests for [Success|Failure]Response could confirm the behavior of .bind. I spent the most time digesting [Success|Failure]Response.bind, and a unit test case or two might've made their behavior plain to see.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I'll add some tests. bind isn't the most intuitive action.

Comment on lines 54 to 67
it "is returned as a hash with the key 'message'" do
expect(failure.message).to eq('baz')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
it "is returned as a hash with the key 'message'" do
expect(failure.message).to eq('baz')
it "is returned as a string" do
expect(failure.message).to eq('baz')

@jvanderhoof jvanderhoof force-pushed the policy-factory-v2 branch 2 times, most recently from a58be64 to 95a15fe Compare August 7, 2023 17:38
@failure = ::FailureResponse
end

def call(factory_template:, request_body:, account:, authorization:)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method call has a Cognitive Complexity of 13 (exceeds 5 allowed). Consider refactoring.

@failure = ::FailureResponse
end

def call(factory_template:, request_body:, account:, authorization:)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method call has 43 lines of code (exceeds 25 allowed). Consider refactoring.

TEMPLATE
end

def data
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method data has 48 lines of code (exceeds 25 allowed). Consider refactoring.

TEMPLATE
end

def data
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method data has 48 lines of code (exceeds 25 allowed). Consider refactoring.

TEMPLATE
end

def data
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method data has 26 lines of code (exceeds 25 allowed). Consider refactoring.

john-odonnell
john-odonnell previously approved these changes Aug 15, 2023
Copy link
Contributor

@john-odonnell john-odonnell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

TEMPLATE
end

def data
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method data has 34 lines of code (exceeds 25 allowed). Consider refactoring.

TEMPLATE
end

def data
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method data has 34 lines of code (exceeds 25 allowed). Consider refactoring.

TEMPLATE
end

def data
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method data has 34 lines of code (exceeds 25 allowed). Consider refactoring.

TEMPLATE
end

def data
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method data has 38 lines of code (exceeds 25 allowed). Consider refactoring.

@success.new(factories)
end

def find(kind:, id:, account:, role:, version: nil)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method find has a Cognitive Complexity of 6 (exceeds 5 allowed). Consider refactoring.

This commit includes the functional code and tests for the Policy
Factory feature.
This commit includes an initial set of Factory templates. These may need
some work before the official release.
@failure = ::FailureResponse
end

def find_all(account:, role:)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method find_all has 26 lines of code (exceeds 25 allowed). Consider refactoring.

@success.new(factories)
end

def find(kind:, id:, account:, role:, version: nil)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Method find has 26 lines of code (exceeds 25 allowed). Consider refactoring.

end

task load: :environment do
binding.pry
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove debugger entry point binding.pry.

end

task test: :environment do
binding.pry
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove debugger entry point binding.pry.

"password": {
"description": "Database Password",
"type": "string"
},
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid comma after the last item of a hash.

@codeclimate
Copy link

codeclimate bot commented Aug 15, 2023

Code Climate has analyzed commit b30637d and detected 114 issues on this pull request.

Here's the issue category breakdown:

Category Count
Complexity 71
Duplication 2
Style 41

The test coverage on the diff in this pull request is 89.6% (50% is the threshold).

This pull request will bring the total coverage in the repository to 88.5% (0.2% change).

View more on Code Climate.

@jvanderhoof jvanderhoof merged commit b63f73d into master Aug 15, 2023
@jvanderhoof jvanderhoof deleted the policy-factory-v2 branch August 15, 2023 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants