what creates SFTPCustomAuthLambdaFunction.zip? 03-sftp-endpoint stack fails

[onescore-agray]

I'm on step 3 in creating the 03-sftp-endpoint stack.
The error is when creating the SFTPCustomAuthLambda resource.

The code for that lambda says it lives here:

      Code:
        S3Bucket:
          !Join [
            "-",
            [
            !FindInMap ["SourceCode", "General", "S3Bucket"],
            Ref: "AWS::Region",
            ],
          ]
        S3Key:
          !Join [
            "/lambda/",
            [
            !FindInMap ["SourceCode", "General", "KeyPrefix"],
            "SFTPCustomAuthLambdaFunction.zip",
            ],
          ]

But I can' find any instructions or shell scripts on what creates and uploads that zip file.

Any idea?

[thegreatgidsby]

@onescore-agray Did this ever get resolved? I am also experiencing the same issue when deploying 03-sftp-endpoint stack where the same resource, SFTPCustomAuthLambda resource is throwing a 403 - access denied error when deploying the template.

[onescore-agray]

@GideonBelete Unfortunately we gave up on this solution, and we never came up with our own solution. We just gave up.

In summary, I do NOT recommend using AWS’s solution for this.

These are all the pitfalls I went through:

The main solution page gives a pretty good overview of the project: https://aws.amazon.com/solutions/implementations/web-client-for-aws-transfer-family/
It looks pretty straightforward, and it gives a pretty good overview, but that is when things go south. It says to examine the code on github which should contain details on how to roll this out: link. On that github page, it has a brief overview with a link to “You can find step-by-step implementation guide to deploy this solution here:”, which takes you BACK to the original solution page. Great. Infinite loop. Where are the step-by-step instructions?

I realize that the main branch (even though has NEWER commits) is not the correct branch to use. There is an open-source branch that has more detailed instructions in the readme (even though it doesn’t have the NEWER commits from the main branch.

OK. There are 8 Cloudformation Stacks to roll out. One of the stacks creates a VPC, so we can’t do that. I can hack it up though. There is no documentation on how to create the lambda zip files. I posted here but no one responded. I see that other github users have created forks from this github repo, and only 1 person has figured out how to create the zips.

There are also 5 shell scripts that need to run, and each script has to be modified independently. 3 Buckets are created, 2 different ACM certs are needed (one for the ALB, one CDN), and one for the lambda function that must live in us-east-1 for a requirement of the CDN.

There is a section to install NodeJS/Angular, but that is not even needed, since it’s supposed to build a docker container. I do not need to install any of that junk on my machine, so all those steps can just be ignored.

I got as far as creating 6 out of 8 CF stacks successfully, and the 7th stack looks to be a pain. It fails with invalid ACM certs which is a pain to debug. Stack 8 is optional, but it sends usage metrics to a remote website, which we absolutely do not want.

There are also .DS_Store files, *.py-e files in this repo, all of which shouts out that this project was hastily created.

I think if we want a front end to SFTP, we roll out our own and not use this broken, overly complex, undocumented mess.

ONE MORE reason not to use this:

• I just found it messes with the GLOBAL settings of API Gateway: https://us-west-2.console.aws.amazon.com/apigateway/home?region=us-west-2#/settings.
  This was the reason our Prod API Gateway logs IAM role kept on getting tweaked. (took a while to figure this out)

My solution would be to:

create an amplify project that sets up the front end. (the 30s limit may limit us on large uploads though)

Use lambda functions to print out the directory in an s3 bucket based on a mapping of user-to-directory in a bucket.

[thegreatgidsby]

@onescore-agray Thanks for the reply- very informative and I appreciate your perspective on this. I did some digging following your reply and noticed that using the main branch following running the ./build.sh.. script that under ./dist/source/lambda there is a SFTPCustomAuthLambdaFunction.py file.

After some looking around, it seems we can convert this .py to a .zip file as mentioned here: Upload local file on parameter ZipFile AWS::Lambda::Function or here: Can I update AWS Lambda function using CloudFormation template?.

However, it seems both of these examples require uploading the .zip to an s3 bucket (assuming we can dump it into the bucket we create when uploading the 03-sftp-endpoint.template CloudFormation template.

I haven't had a chance to dig further into this or try anything new just yet but I plan to go through the 03-sftp-endpoint.template file to understand what exactly is happening there- the .py file could have already been integrated as part of the distribution so I will need to look into it further. In addition, initially creating the dist folder locally takes creates the s3 bucket name to replace the %%BUCKET_NAME%% (sp?) variables in the .template.

I think there is an opportunity to considerably simplify this process or at least a chance to provide clear deployment instructions so I will update as I work through it.

Thank you again for your reply- it provided me a path to start exploring.

[thegreatgidsby]

@onescore-agray Just following up but I resolved the issue. I will edit this comment later with a write-up on how to resolve the problem which largely stems from a lack of implementation documentation (or availability of any implementation documentation on the 'main' branch as you mentioned). I will look to create a pull request and submit a proper implementation write up once I have completed full implementation to ensure there are no other issues which I can then include. For now, I will edit this comment later when I have a chance with a solution on implementing 03-sftp-endpoint.template in case you were still looking to use it at all.

[andersontechnetverrum]

Hi, @thegreatgidsby am facing the same issues you mentioned above due to lack of implementation.

I was wondering if you could push your PR to be reviewed so I could implement it as part of my testing env.

Thank you so much in advance for sharing your thoughts here.

[thegreatgidsby]

@andersontechnetverrum apologies for the delayed response. I am in the process of preparing and finalizing my revised documentation for this implementation and will open a PR shortly if you were still looking for a solution.

[yarozen]

Hi @thegreatgidsby, any update on this?
I am facing the same issue.