AWS-SNS Configuration

[dkusek]

I have not been able to get this to send to an email endpoint.

My configuration:
Region: us-east-2
ARN: arn:sns:us-east-2.......................

The protocol on the AWS side is through "email".

Thats all correct?
Is there anything else in networking that should be checked? I can send messages directly from AWS-SNS to the endpoint so I dont think they are being blocked. I also messed around with the newest nightly that has the ability to send "Test Alerts" and that was also unsuccessful. I went one more and set an additional region, us-east-1, but that had the same outcome. Any help on this is greatly appreciated. Unless others are having problems, I must be missing something small.

EDIT: I have also set up SQS in Amazon and Email-JSON. I can send messages within the SQS service no problem and I can send the email-json messages to the email endpoint. Again, nothing seems to be wrong with the functionality. Is there any additional settings on the FreeNAS side for this to be functional?

 

[dkusek]

I did a bit more poking around on the AWS-SNS side. Does this alert service within FreeNAS need to have some kind of signature or key issued on the AWS side of things? I have also looked into permissions on the AWS side. I opened them completely up (obviously security will prohibit me from doing this long term but did it for testing) and I got nothing. I feel this feature is pretty straightforward. If any who has the alert service work with AWS-SNS could post an example of what their config is and what the AWS-SNS security/permission settings need to be (if anything outside of default), that would be great.

Thank you.

 

[dkusek]

So just wanted to report, I updated to 11.0-U1 and still not change. Im currently working in a VM. Setting up a dummy "tank" and then removing one or two of the drives in order to trigger an alert. I get nothing on the SNS side. It would be extremely helpful to see if anyone has gotten this to work. At least then, I would know whether it is on my side of things or not as far as the SNS settings go. My ideal use case is to have a single SNS account forward alert sent by different storage servers in order to manage alerts on one specific email address. I know this can be configured through just setting up email alerts in the GUI, but the security layers on the email server side of things prohibit me from doing so. Also, configuring the alert service should be much easier as its just a couple of boxes (unless I am missing something of course). And on that last note, yes, I have set up email alerts before through the specific email tab in the GUI. That is not a route I wish to use if at all possible.

 

[dlavigne]

Do you get an errors in /var/log/messages when you try to send?

 

[dkusek]

No errors are reported in /var/log/messages when "Send Test Alert" is selected. Furthermore, no error logs are reported when I pull a physical drive on a test system to actually generate a real error. Thoughts?

 

[dlavigne]

It may be a bug then. Please create a report at bugs.freenas.org that includes the above info and post the issue number here.

 

[dkusek]

https://bugs.freenas.org/issues/25037

Bug# 25037

Thank you for the assistance!

 

[Gunnar Grim]

Perhaps the bug report could also mention that there is no way to specify the AWS access key ID or AWS secret access key.

 

[dkusek]

Thank you Gunnar. I put that in the bug report.

 

[dkusek]

Im also assuming that from the lack of responses that AWS-SNS does not function for anyone? Forgive me, but how do things get into the final 11.0 OS AND get documented in both the blog and documentation that DONT WORK? I am not one to point fingers. But is there QA being done? Or did no one outside of the two people on this forum try to use AWS-SNS for the alert service.... I will shoulder some of the blame as I should have taken the initiative and tested this on one of the RC's that were out prior to 11.0. Im just confused by all of this. From the surface, it truly looks like the button is there... but everything else is non-functional.

Again, has anyone gotten the Amazon AWS-SNS alert service to work? Has anyone gotten any of the other alert service methods to work?

 

[Steve Andrews]

Slack alerts have not worked for me. Had an issue that I think should have generated something (critical: boot volume degraded, got email alert).

 

[Marcelo Araujo]

Im also assuming that from the lack of responses that AWS-SNS does not function for anyone? Forgive me, but how do things get into the final 11.0 OS AND get documented in both the blog and documentation that DONT WORK? I am not one to point fingers. But is there QA being done? Or did no one outside of the two people on this forum try to use AWS-SNS for the alert service.... I will shoulder some of the blame as I should have taken the initiative and tested this on one of the RC's that were out prior to 11.0. Im just confused by all of this. From the surface, it truly looks like the button is there... but everything else is non-functional.

Again, has anyone gotten the Amazon AWS-SNS alert service to work? Has anyone gotten any of the other alert service methods to work?

There is no lack of response, myself as a developer have tickets that I'm working on too. I don't have only your ticket.

Yes we have QA and we did test as much as possible with the consul alerts and with different providers.

So, I'm checking what could be the issue. Also attache your debug files, it can be useful.

Thanks,

 

[dkusek]

I meant response from other individuals trying to use the service to understand if this one on my side or widespread. I understand what support and development entails. Per this situation, my debug file will be the same as anyone else's that have tried to use AWS-SNS with no avail. Please understand that I have been working exclusively with FreeNAS in a multiple user/ multiple machine stance for some time now so I understand what networking functions need to be in place. I dedicated a week of my time solely on this issue so im fairly confident that this is not a "user error." I have eaten my words before though.... The mere fact that it generates nothing in the log messages is a clear sign to me that the implementation is off somewhere.

I have found a workaround by using AWS-SES instead and the proven email alert method time tested on FreeNAS. It is actually a better route to go in that AWS-SES gives you the ability to create multiple SMTP users/password and manage them through IAM so that a person managing multiple machines can have them go to one location and bounce to an alternate address. It is more secure and SMTP users can be deactivated if necessary in the event of some kind of breach per that user's account.

 

[entilza72]

+1

I am brand new to FreeNAS so might be prone to getting things wrong. Google searching after my identical experience brought me here:

I configured the AWS-SNS service on the AWS side
I plugged in the ARN and Region (NB, not sure what purpose the region field serves - can't you get that from the ARN string?)

No output. No errors. AWS is unhelpful here though because they usually do not report access attempts. I tested the SMS sending on AWS side and it was fine.

Entilza.

 

[hendry]

I've hit this issue. The bug is marked resolved. https://redmine.ixsystems.com/issues/25037 I am bit puzzled and disheartened.

 

[dlavigne]

Please create a new bug with a debug attached. Mention the old bug number in your report.

 

[hendry]

Apologies for the noise. I did just receive a notification.

Code:

freenas.local:fake-27806:Service 'fake-27806' check is critical.

Surprised to see nothing in the log files about this.

 

[kongping]

I tried SNS api and unexpectly stucked at authorization part for 2 DAYS!!!
first I thought it was suppose to be Signature Version 2 because most examples out there are so, but somehow it didn't work so I tried Version 4. After reading through many complex and mind crushing documentation and forum, I found one example of python and s3.

I have changed it to SNS, tried and it works. (remember to use your own credential and region)
I tried to share it on AWS forum, but only premium users can post, so I just randomly register in a forum and post it. lol


import sys, os, base64, datetime, hashlib, hmac, urllib
import requests

# Request values
method = 'GET'
service = 'sns'
host = 'sns.us-east-1.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://sns.us-east-1.amazonaws.com'
# date for headers and credential string
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d')
# credentials
access_key = "xxxxxxxxxx"
secret_key = "yyyyyyyyyy"

# Functions
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning

# ************* TASK 1: CREATE A CANONICAL REQUEST *************
canonical_uri = '/'
canonical_headers = 'host:' + host + '\n'
signed_headers = 'host'
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'

# Create canonical query string. must be URL-encoded (space=%20)
# sorted by name, use urllib.parse.quote_plus() if using Python 3
canonical_querystring = 'Action=CreateTopic&Name=My-Topic&Version=2010-03-31'
canonical_querystring += '&X-Amz-Algorithm=AWS4-HMAC-SHA256'
canonical_querystring += '&X-Amz-Credential=' + urllib.parse.quote_plus(access_key + '/' + credential_scope)
canonical_querystring += '&X-Amz-Date=' + amz_date
canonical_querystring += '&X-Amz-Expires=86400'
canonical_querystring += '&X-Amz-SignedHeaders=' + signed_headers

payload_hash = hashlib.sha256(('').encode('utf-8')).hexdigest()
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash

# ************* TASK 2: CREATE SIGNING STRING************
string_to_sign = algorithm + '\n' + amz_date + '\n' + credential_scope + '\n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()

# ************* TASK 3: CALCULATE SIGNATURE *************
signing_key = getSignatureKey(secret_key, datestamp, region, service)
signature = hmac.new(signing_key, (string_to_sign).encode("utf-8"), hashlib.sha256).hexdigest()

# ************* TASK 4: ADD SIGNING INFO TO REQUEST *************
canonical_querystring += '&X-Amz-Signature=' + signature

# ************* SEND THE REQUEST *************
request_url = endpoint + "?" + canonical_querystring
# print(request_url)

print('Request URL = ' + request_url)
r = requests.get(request_url)
print('Response code: %d\n' % r.status_code)
print(r.text)