Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

custodian run doesn't allow (AWS) --region=all with non-default --profile #9433

Open
dannysauer opened this issue Apr 12, 2024 · 0 comments
Open
Labels

Comments

@dannysauer
Copy link
Contributor

Describe the bug

It appears that using the --profile argument breaks the --regions all mode.

custodian run --profile id_NNN --region all ... results in botocore.exceptions.NoRegionError: You must specify a region.

It works fine if I use a specific region or multiple regions, but all does not work. Alternatively, using the credentials associated with default (defined in .aws/credentials) instead of an alternate profile works as expected. For whatever it adds, this also fails if I export AWS_PROFILE=id_NNN beforehand as well instead of using the --profile custodian argument.

Without looking at the code, I'd guess this to be an issue with the config's default region not being picked up as expected when custodian initially gets the list of regions to iterate? The AWS CLI seems to pick it up, but maybe boto isn't respecting the region value from source_profile?

My ~/.aws/config looks generally like this:

[profile default]
region = us-east-2
output = json
...
[profile id_NNN]
source_profile = default
role_arn = arn:aws:iam::NNN:role/OrganizationAccountAccessRole

What did you expect to happen?

I expected the policy to run in all regions the account specified by the profile.

Cloud Provider

Amazon Web Services (AWS)

Cloud Custodian version and dependency information

Custodian:   0.9.34
Python:      3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
Platform:    posix.uname_result(sysname='Linux', nodename='fabulinus', release='6.8.0-76060800daily20240311-generic', version='#202403110203~1711393930~22.04~331756a SMP PREEMPT_DYNAMIC Mon M', machine='x86_64')
Using venv:  True
Docker: False
Installed: 

PyJWT==2.8.0
adal==1.2.7
anyio==4.2.0
applicationinsights==0.11.10
apscheduler==3.10.4
argcomplete==3.2.1
attrs==23.2.0
azure-common==1.1.28
azure-core==1.29.6
azure-cosmos==3.2.0
azure-cosmosdb-nspkg==2.0.2
azure-cosmosdb-table==1.0.6
azure-functions==1.18.0
azure-graphrbac==0.61.1
azure-identity==1.15.0
azure-keyvault==4.2.0
azure-keyvault-certificates==4.7.0
azure-keyvault-keys==4.8.0
azure-keyvault-secrets==4.7.0
azure-mgmt-advisor==9.0.0
azure-mgmt-apimanagement==1.0.0
azure-mgmt-appconfiguration==0.7.0
azure-mgmt-applicationinsights==1.0.0
azure-mgmt-appplatform==8.0.0
azure-mgmt-authorization==1.0.0
azure-mgmt-automation==0.1.1
azure-mgmt-batch==15.0.0
azure-mgmt-cdn==12.0.0
azure-mgmt-cognitiveservices==11.0.0
azure-mgmt-compute==19.0.0
azure-mgmt-containerinstance==7.0.0
azure-mgmt-containerregistry==8.0.0b1
azure-mgmt-containerservice==15.1.0
azure-mgmt-core==1.4.0
azure-mgmt-cosmosdb==6.4.0
azure-mgmt-costmanagement==1.0.0
azure-mgmt-databricks==1.0.0b1
azure-mgmt-datafactory==1.1.0
azure-mgmt-datalake-analytics==0.5.0
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.5.0
azure-mgmt-desktopvirtualization==1.1.0
azure-mgmt-dns==8.0.0b1
azure-mgmt-eventgrid==8.0.0
azure-mgmt-eventhub==11.0.0
azure-mgmt-frontdoor==1.1.0
azure-mgmt-hdinsight==7.0.0
azure-mgmt-iothub==1.0.0
azure-mgmt-keyvault==8.0.0
azure-mgmt-kusto==2.2.0
azure-mgmt-logic==9.0.0
azure-mgmt-machinelearningservices==1.0.0
azure-mgmt-managementgroups==1.0.0b1
azure-mgmt-monitor==2.0.0
azure-mgmt-msi==1.0.0
azure-mgmt-network==17.1.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-policyinsights==1.0.0
azure-mgmt-rdbms==8.1.0
azure-mgmt-recoveryservices==2.5.0
azure-mgmt-recoveryservicesbackup==7.0.0
azure-mgmt-redhatopenshift==1.4.0
azure-mgmt-redis==12.0.0
azure-mgmt-resource==16.1.0
azure-mgmt-resourcegraph==7.0.0
azure-mgmt-search==8.0.0
azure-mgmt-security==1.0.0
azure-mgmt-servicebus==8.2.0
azure-mgmt-servicefabric==1.0.0
azure-mgmt-signalr==0.4.0
azure-mgmt-sql==1.0.0
azure-mgmt-storage==17.1.0
azure-mgmt-streamanalytics==1.0.0
azure-mgmt-subscription==1.0.0
azure-mgmt-synapse==2.0.0
azure-mgmt-trafficmanager==0.51.0
azure-mgmt-web==2.0.0
azure-nspkg==3.0.2
azure-storage-blob==12.19.0
azure-storage-common==2.1.0
azure-storage-file==2.1.0
azure-storage-file-share==12.15.0
azure-storage-queue==12.9.0
boto3==1.34.21
botocore==1.34.21
c7n==0.9.34
cachetools==5.3.2
certifi==2023.11.17
cffi==1.16.0
charset-normalizer==3.3.2
click==8.1.7
cryptography==41.0.7
distlib==0.3.8
docutils==0.18.1
exceptiongroup==1.2.0
google-api-core==2.15.0
google-api-python-client==2.114.0
google-auth==2.26.2
google-auth-httplib2==0.2.0
google-cloud-appengine-logging==1.4.0
google-cloud-audit-log==0.2.5
google-cloud-core==2.4.1
google-cloud-logging==3.9.0
google-cloud-monitoring==2.18.0
google-cloud-storage==2.14.0
google-crc32c==1.5.0
google-resumable-media==2.7.0
googleapis-common-protos==1.62.0
grpc-google-iam-v1==0.13.0
grpcio==1.60.0
grpcio-status==1.60.0
httplib2==0.22.0
idna==3.6
importlib-metadata==6.11.0
isodate==0.6.1
jmespath==1.0.1
jsonschema==4.21.0
jsonschema-specifications==2023.12.1
msal==1.26.0
msal-extensions==1.1.0
msrest==0.7.1
msrestazure==0.6.4
netaddr==0.7.20
oauthlib==3.2.2
packaging==23.2
portalocker==2.8.2
proto-plus==1.23.0
protobuf==4.25.2
pyasn1==0.5.1
pyasn1-modules==0.3.0
pycparser==2.21
pyjwt==2.8.0
pyparsing==3.1.1
pyrate-limiter==2.10.0
python-dateutil==2.8.2
pytz==2023.3.post1
pyyaml==6.0.1
referencing==0.31.1
requests==2.31.0
requests-oauthlib==1.3.1
retrying==1.3.4
rpds-py==0.17.1
rsa==4.9
s3transfer==0.10.0
six==1.16.0
sniffio==1.3.0
tabulate==0.9.0
typing-extensions==4.9.0
tzlocal==5.2
uritemplate==4.1.1
urllib3==1.26.18
zipp==3.17.0

Policy

policies:
  - name: find-eks-clusters
    resource: aws.eks

Relevant log/traceback output

Traceback (most recent call last):
  File "/home/sauer/venv/custodian/bin/custodian", line 8, in <module>
    sys.exit(main())
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/c7n/cli.py", line 363, in main
    command(config)
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/c7n/commands.py", line 99, in _load_policies
    policies += provider.initialize_policies(
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/c7n/resources/aws.py", line 772, in initialize_policies
    get_profile_session(options).client('ec2').describe_regions(
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/boto3/session.py", line 299, in client
    return self._session.create_client(
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/botocore/session.py", line 997, in create_client
    client = client_creator.create_client(
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/botocore/client.py", line 161, in create_client
    client_args = self._get_client_args(
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/botocore/client.py", line 508, in _get_client_args
    return args_creator.get_client_args(
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/botocore/args.py", line 100, in get_client_args
    final_args = self.compute_client_args(
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/botocore/args.py", line 219, in compute_client_args
    endpoint_config = self._compute_endpoint_config(
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/botocore/args.py", line 369, in _compute_endpoint_config
    return self._resolve_endpoint(**resolve_endpoint_kwargs)
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/botocore/args.py", line 474, in _resolve_endpoint
    return endpoint_bridge.resolve(
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/botocore/client.py", line 613, in resolve
    resolved = self.endpoint_resolver.construct_endpoint(
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/botocore/regions.py", line 229, in construct_endpoint
    result = self._endpoint_for_partition(
  File "/home/sauer/venv/custodian/lib/python3.10/site-packages/botocore/regions.py", line 277, in _endpoint_for_partition
    raise NoRegionError()
botocore.exceptions.NoRegionError: You must specify a region.

Extra information or context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant