We are excited to announce the brand new custom field functionality in Reporter! This powerful new feature gives you the flexibility to tailor your Reporter instance like never before. You can now add various types of custom fields to findings, assessments, clients, and users to meet your specific needs.
In this blog post, we’ll demonstrate some of these new features by building a basic integration with GitHub. Linking assessments to an issue-tracking system like GitHub enables developers to quickly act on findings created by researchers in Reporter. While this example focuses on GitHub, the concepts covered can be applied to integrate with other popular platforms, such as Jira.
To achieve this, we’ll use Flask to handle webhook requests from both Reporter and GitHub. When a new finding is created in Reporter, it automatically generates a corresponding issue in GitHub. Any updates to the finding in Reporter are synced with the GitHub issue, keeping information consistent across both systems. Finally, when the issue is closed in GitHub, the finding’s review status in Reporter updates to 'Revision Requested Review'.
Creating an integration yourself using APIs and webhooks offers far more flexibility than a built-in integration. You can define exactly which actions should occur under specific conditions, ensuring the solution is perfectly tailored to your workflow. In a follow-up blog post, we’ll show you how to create this same integration without writing any code, using Zapier.
Prerequisites
- A running instance of Reporter
- A Python 3 (virtual) environment with the following libraries installed:
- Flask - a lightweight web application framework
- PyGithub - a Python library to access the GitHub API
- securityreporter - our Python library for accessing the Reporter API
- A (sub)domain that can be used to access the Flask application. For development purposes you can consider using a service such as ngrok.
Step 1: Set up the custom fields
Log in to Reporter as an admin user. Navigate to Settings > Custom fields. We’ll create three custom fields:
- First, create a new 'Number' field for the 'Finding' model to store the GitHub issue number. Name the field
c_github_issue, which is automatically done by setting the label to 'Github issue'. In the field permissions, ensure admin users have both read and write access.
- Next, add a 'URL' field for the 'Finding' model to store the GitHub issue URL. Name this field
c_github_issue_url. - Finally, create a 'Text' field for the 'Assessment' model to store the GitHub repository. Name this field
c_github_repo.
Step 2: Set up a webhook in Reporter
Next, navigate to Settings > Webhooks and create a new webhook. Set the URL to https://{domain_of_your_flask_app}/reporter-webhook. Generate a secret that you store in a safe place. Under Webhook Types select finding:updated, and set the Includes field to assessment so that assessment data is sent along with finding data when the webhook is triggered. You can use the Preview data button to preview the contents of such a webhook request.
Step 3: Create an API token in Reporter
Navigate to Settings > API tokens and create a new token. The only permissions you need in this example are Read on Assessments and Read and write on Findings. You’ll receive an API token; store this token in a safe place.
Step 4: Set up your GitHub repository
You’ll need at least one GitHub repository where issues are stored. Go to the GitHub repository settings and create a webhook. Set the payload URL to https://{domain_of_your_flask_app}/github-webhook. Generate a secret and store it safely. Select that the webhook should be triggered by individual events, and only select Issues events.
Step 5: Obtain an access token from GitHub
Go to the GitHub developer settings, and create a new fine-grained personal access token. Only give it access to the repository you used in Step 4. Under Repository Access, give it Read and write access to Issues. Store the access token in a safe place.
Step 6: Deploy the Flask application
The following code sets up a Flask application that listens on the /reporter-webhook and /github-webhook endpoints. It ensures the authenticity of these requests by verifying the message signatures using the secrets you obtained when creating the webhooks.
You can extend the application to support more types of events by adding them to the reporter_webhook() and github_webhook() methods.
Before running the application, update the configuration variables at the top of the code. 🔒Tip: For production environments, avoid storing secrets directly in your application code. Instead, use environment variables or separate configuration files for better security.
To run the application in development, execute the following command: flask --app app.py --debug run. This starts a Flask process that listens to port 5000. For deploying your application in a production environment, you can choose from various deployment strategies. Refer to the Flask documentation for more details.
from flask import Flask, request, Response, abort, jsonify
from github import Auth, Github
from reporter import Reporter
import hmac
import hashlib
app = Flask(__name__)
# Settings - change before using
TARGET_LABEL='security reporter'
REPORTER_TOKEN='<Reporter API token>'
REPORTER_URL='<Reporter URL>'
GITHUB_TOKEN='<GitHub API token>'
REPORTER_WEBHOOK_SECRET= '<Reporter webhook secret>'
GITHUB_WEBHOOK_SECRET='<Github webhook secret>'
rc = Reporter(url=REPORTER_URL, api_token=REPORTER_TOKEN, ssl_verify=False)
auth = Auth.Token(GITHUB_TOKEN)
g = Github(auth=auth)
@app.route('/reporter-webhook', methods=['POST'])
def reporter_webhook():
"""Endpoint that handles webhooks coming from Reporter"""
verify_webhook_signature('Signature', REPORTER_WEBHOOK_SECRET)
payload = request.json
if payload.get('webhook_type') == 'finding:updated':
update_github_issue_from_finding(payload['model'])
return Response()
@app.route('/github-webhook', methods=['POST'])
def github_webhook():
"""Endpoint that handles webhooks coming from GitHub"""
verify_webhook_signature('X-Hub-Signature-256', GITHUB_WEBHOOK_SECRET, after="sha256=")
event = request.headers.get('X-GitHub-Event')
payload = request.json
if event == 'issues' and payload['action'] == 'closed':
labels = [label['name'] for label in payload['issue'].get('labels', [])]
if TARGET_LABEL in labels:
set_finding_to_retest(payload)
return Response()
def update_github_issue_from_finding(finding):
"""If a finding is updated, make a new issue on Github or update the existing one. Needs `assessment` to be included in webhook data!"""
if finding['assessment']['c_github_repo'] is None:
return
issue_number = finding['c_github_issue']
repo = g.get_repo(finding['assessment']['c_github_repo'])
if issue_number is None:
# If there is no related issue attached to this finding, create a new issue and attach it to the finding.
issue = repo.create_issue(title=finding['title'], body=finding['description'], labels=[TARGET_LABEL])
rc.findings.update(finding['id'], {
'c_github_issue': issue.number,
'c_github_issue_url': issue.html_url
})
else:
repo.get_issue(number=issue_number).edit(title=finding['title'], body=finding['description'])
def set_finding_to_retest(payload):
"""When a GitHub issue is closed, set the review status of all corresponding findings to Retest"""
assessments = rc.assessments.list(filter={
'c_github_repo': payload['repository']['full_name']
})
findings = rc.findings.list(filter={
'assessment_id': [assessment.id for assessment in assessments],
'c_github_issue': payload['issue']['number']
}, include = 'assessment')
for finding in findings:
if finding.review_status != 3: # Avoid infinite loop of webhooks
rc.findings.update(finding.id, {'review_status': 3})
@app.after_request
def add_reporter_header(response):
"""Adds a `Reporter` header to each response"""
response.headers["Reporter"] = ""
return response
def abort_with_message(message, status=403):
response = jsonify(message=message)
response.status = status
abort(response)
def verify_webhook_signature(header_name, secret, after=""):
signature = request.headers.get(header_name)
if signature is None:
abort_with_message(f"The {header_name} header is missing", 403)
mac = hmac.new(secret.encode('utf-8'), request.get_data(), digestmod=hashlib.sha256)
if not hmac.compare_digest(after + mac.hexdigest(), signature):
abort_with_message("The webhook signature is invalid", 403)
if __name__ == '__main__':
app.run()
Step 7: Try it out!
In Reporter, open one of your assessments and edit it. In the GitHub Repo field, enter the GitHub repository in the {owner}/{repository} format. Next, go to a finding and update its description.
If everything is set up correctly, an issue will be created in your GitHub repository, and a link to this GitHub issue will appear in the corresponding finding in Reporter. Closing the issue in GitHub will automatically update the finding's status in Reporter to 'Revision Requested'.
The finding in Reporter:
The issue in GitHub:
