Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make changes for a public release #105

Merged
merged 44 commits into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from 43 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
09aaa94
Create a publish.yml file to publish to PyPI
Aug 20, 2024
24ce4f0
Remove debug logging for e.g. builtins, repetitive useless statements
Aug 20, 2024
1fbf00a
Downgrade to debug logs
Aug 20, 2024
65615f4
Make messages more concise or add better data to logs
Aug 20, 2024
3772693
Aikido RASP to Aikido Zen
Aug 20, 2024
fdf15c6
Merge branch 'main' into publish-to-pypi
Aug 20, 2024
e8396fe
Merge branch 'AIK-3348' into publish-to-pypi
Aug 20, 2024
312fdbd
Merge branch 'AIK-3350' into publish-to-pypi
Aug 20, 2024
02c45a7
v0.1.0
Aug 20, 2024
b109435
Fix failed to report an attack broken test
Aug 20, 2024
b5f1bf6
remove leftover /legacy/ url from TestPyPI
Aug 20, 2024
7d266cb
Revert package name to aikido_firewall
Aug 20, 2024
3daadb5
Use twine to publish
Aug 20, 2024
a73b508
Remove python setup.py command
Aug 20, 2024
5594c86
Merge branch 'AIK-3337' into publish-to-pypi
Aug 21, 2024
0e41117
Use guard.aikido.dev and v0.1.1
Aug 21, 2024
2353b4c
Merge branch 'AIK-3348' into AIK-3305
Aug 21, 2024
b15758a
Add hasReceivedAnyStats, if shouldSendStat schedule heartbeat
Aug 21, 2024
0b0949e
update_service_config check if res.endpoints is not none + linting
Aug 21, 2024
96bc3ca
Add a boot argument to the heartbeats
Aug 21, 2024
70c3bb2
Execute the report_initial_statistics only after 60 seconds
Aug 21, 2024
291c5aa
Changed start_polling_for_changes to use reporter object
Aug 21, 2024
d015440
Update tests for start_polling_for_changes and the service_config
Aug 21, 2024
42e463f
add is_empty function to Statistics class
Aug 21, 2024
4ff47c4
Use is_empty function and cleanup logging
Aug 21, 2024
e651767
Merge branch 'AIK-3305' into publish-to-pypi
Aug 21, 2024
cdf37b3
Move to version 0.1.2
Aug 21, 2024
16a3cf8
Send KILL when SIGINT/SIGTERM
Aug 22, 2024
feba796
Merge branch 'main' into publish-to-pypi
Aug 22, 2024
1889b63
Version 0.1.3
Aug 22, 2024
20ce8c0
Update the way we kill the background process
Aug 22, 2024
5b429e8
0.1.4
Aug 22, 2024
66ef031
Use atexit to exit
Aug 22, 2024
cbc2935
Replace 0.1.4 with 0.1.5
Aug 22, 2024
2bb0390
0.1.6 (Another signal_handler method)
Aug 22, 2024
60b9227
Linting
Aug 22, 2024
b46d291
0.1.7 (Another attempt at killing bg process)
Aug 22, 2024
8e73f5c
v0.1.8 : Remove signal_handler , remove protect server=False
Aug 22, 2024
2729bd7
Run a KILL to make sure no bg processes no more
Aug 22, 2024
f16d706
0.1.9
Aug 22, 2024
78cc80b
Merge branch 'main' into publish-to-pypi
Aug 22, 2024
b6ffc5b
Reset to 0.0.9
Aug 22, 2024
f017156
Reset changes made to SIGTERM/SIGINT stuff
Aug 22, 2024
23bf19a
Only publish on release and not on tag
Aug 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
name: Publish Python 🐍 distribution 📦 to PyPI

on:
push:
tags:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on release zoals bij node

- 'v*'

jobs:
tests:
name: Linting and running tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install poetry
run: pip install poetry
- name: Install dependencies
run: poetry install
- name: Run Pylint
run: |
poetry run pylint --fail-under=9 --rcfile=.pylintrc aikido_firewall/
pylint_exit_code=$?
if [ $pylint_exit_code -ne 0 ]; then
echo "Pylint check failed. Please fix the issues."
exit 1
fi

- name: Run Black Check
run: |
poetry run black --check --diff aikido_firewall/
black_exit_code=$?
if [ $black_exit_code -ne 0 ]; then
echo "Black check failed. Please run 'black .' to format the code."
exit 1
fi
- name: Run tests
run: |
make test
build:
name: Build distribution 📦
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Install poetry
run: pip install poetry
- name: Install dependencies
run: poetry install

- name: Build distribution packages
run: poetry build

- name: Store the distribution packages
uses: actions/upload-artifact@v3
with:
name: python-package-distributions
path: dist/
publish-to-pypi:
name: Publish Python 🐍 distribution 📦 to PyPI
needs:
- build
- tests
runs-on: ubuntu-latest

permissions:
id-token: write # IMPORTANT: mandatory for trusted publishing

steps:
- name: Download all the dists
uses: actions/download-artifact@v3
with:
name: python-package-distributions
path: dist/
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install setuptools wheel twine

- name: Publish to PyPI
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
run: |
twine upload dist/*
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@
) # Create an event scheduler
self.send_to_reporter(event_scheduler)

api = ReportingApiHTTP("http://app.local.aikido.io/")
api = ReportingApiHTTP("https://guard.aikido.dev/")

Check warning on line 66 in aikido_firewall/background_process/aikido_background_process.py

View check run for this annotation

Codecov / codecov/patch

aikido_firewall/background_process/aikido_background_process.py#L66

Added line #L66 was not covered by tests
# We need to pass along the scheduler so that the heartbeat also gets sent
self.reporter = Reporter(
block=check_env_for_blocking(),
Expand All @@ -84,7 +84,6 @@
event_scheduler.enter(
EMPTY_QUEUE_INTERVAL, 1, self.send_to_reporter, (event_scheduler,)
)
logger.debug("Checking queue")
while not self.queue.empty():
queue_attack_item = self.queue.get()
self.reporter.on_detected_attack(
Expand Down
2 changes: 1 addition & 1 deletion aikido_firewall/background_process/commands/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,4 @@
if action in commands_map:
commands_map[action](bg_process, data, conn)
else:
logger.info("Command : `%s` not found, aborting", action)
logger.debug("Command : `%s` not found, aborting", action)

Check warning on line 40 in aikido_firewall/background_process/commands/__init__.py

View check run for this annotation

Codecov / codecov/patch

aikido_firewall/background_process/commands/__init__.py#L40

Added line #L40 was not covered by tests
4 changes: 2 additions & 2 deletions aikido_firewall/background_process/comms.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,8 @@ def target(address, key, receive, data, result_obj):
t.start()
t.join(timeout=0.1)
if not result_obj[0]:
logger.info(
"Communication returned None between background process and threads"
logger.debug(
" Failure in communication to background process, %s(%s)", action, obj
)
return {"success": False, "error": "timeout"}

Expand Down
2 changes: 1 addition & 1 deletion aikido_firewall/background_process/packages.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
try:
pkg_version = metadata.version(pkg_name)
except metadata.PackageNotFoundError:
logger.info(
logger.debug(

Check warning on line 15 in aikido_firewall/background_process/packages.py

View check run for this annotation

Codecov / codecov/patch

aikido_firewall/background_process/packages.py#L15

Added line #L15 was not covered by tests
"Package `%s` was wrapped but could not find a version, aborting", pkg_name
)
return
Expand Down
9 changes: 2 additions & 7 deletions aikido_firewall/background_process/realtime/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,7 @@
}
response = requests.get(url, headers=headers, timeout=3) # timeout in 3 seconds
if response.status_code != 200:
logger.info(
"Invalid response from api (Status Code : %s)", response.status_code
)
logger.info("Invalid response from api : %s", response.status_code)

Check warning on line 28 in aikido_firewall/background_process/realtime/__init__.py

View check run for this annotation

Codecov / codecov/patch

aikido_firewall/background_process/realtime/__init__.py#L28

Added line #L28 was not covered by tests

return response.json() # Parse and return the JSON response

Expand All @@ -42,9 +40,6 @@
}
response = requests.get(url, headers=headers, timeout=0.5) # timeout in 500ms
if response.status_code != 200:
logger.info(
"Invalid response from realtime api (Status Code : %s)",
response.status_code,
)
logger.info("Invalid response from realtime api : %s", response.status_code)

Check warning on line 43 in aikido_firewall/background_process/realtime/__init__.py

View check run for this annotation

Codecov / codecov/patch

aikido_firewall/background_process/realtime/__init__.py#L43

Added line #L43 was not covered by tests

return int(response.json()["configUpdatedAt"]) # Return configUpdatedAt time
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,4 @@ def on_detected_attack(reporter, attack, context, blocked, stack):
logger.debug("Result : %s", result)
except Exception as e:
logger.debug(e)
logger.info("Failed to report attack")
logger.info("Failed to report an attack")
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ def test_on_detected_attack_exception_handling(mock_reporter, mock_context, capl

on_detected_attack(mock_reporter, attack, mock_context, blocked=False, stack=None)

assert "Failed to report attack" in caplog.text
assert "Failed to report an attack" in caplog.text


def test_on_detected_attack_with_blocked_and_stack(mock_reporter, mock_context):
Expand Down
2 changes: 0 additions & 2 deletions aikido_firewall/sinks/builtins.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@

import copy
import importhook
from aikido_firewall.helpers.logging import logger
from aikido_firewall.vulnerabilities import run_vulnerability_scan


Expand All @@ -28,5 +27,4 @@ def aikido_new_open(*args, **kwargs):

# pylint: disable=no-member
setattr(builtins, "open", aikido_new_open)
logger.debug("Wrapped `builtins` module")
return modified_builtins
3 changes: 1 addition & 2 deletions aikido_firewall/sinks/http_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,12 @@
former_putrequest = copy.deepcopy(http.HTTPConnection.putrequest)

def aik_new_putrequest(_self, method, url, *args, **kwargs):
logger.info("HTTP Request [%s] %s:%s %s", method, _self.host, _self.port, url)
logger.debug("HTTP Request [%s] %s:%s %s", method, _self.host, _self.port, url)

Check warning on line 23 in aikido_firewall/sinks/http_client.py

View check run for this annotation

Codecov / codecov/patch

aikido_firewall/sinks/http_client.py#L23

Added line #L23 was not covered by tests
run_vulnerability_scan(
kind="ssrf", op="http.client.putrequest", args=(_self.host, _self.port)
)
return former_putrequest(_self, method, url, *args, **kwargs)

# pylint: disable=no-member
setattr(http.HTTPConnection, "putrequest", aik_new_putrequest)
logger.debug("Wrapped `http` module")
return modified_http
2 changes: 0 additions & 2 deletions aikido_firewall/sinks/os.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@

import copy
import importhook
from aikido_firewall.helpers.logging import logger
from aikido_firewall.vulnerabilities import run_vulnerability_scan

# File functions :
Expand Down Expand Up @@ -68,5 +67,4 @@ def on_os_import(os):
# pylint: disable=no-member
setattr(modified_os.path, op, aikido_new_func)

logger.debug("Wrapped `os` module")
return modified_os
2 changes: 0 additions & 2 deletions aikido_firewall/sinks/os_system.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@

import copy
import importhook
from aikido_firewall.helpers.logging import logger
from aikido_firewall.vulnerabilities import run_vulnerability_scan


Expand All @@ -26,5 +25,4 @@ def aikido_new_system(*args, former_system_func=former_system_func, **kwargs):
setattr(os, "system", aikido_new_system)
setattr(modified_os, "system", aikido_new_system)

logger.debug("Wrapped `os` module")
return modified_os
3 changes: 1 addition & 2 deletions aikido_firewall/sinks/socket.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,12 @@ def on_socket_import(socket):
- gethostbyname() -- map a hostname to its IP number
- gethostbyaddr() -- map an IP number or hostname to DNS info
https://github.com/python/cpython/blob/8f19be47b6a50059924e1d7b64277ad3cef4dac7/Lib/socket.py#L10
Returns : Modified http.client object
Returns : Modified socket object
"""
modified_socket = importhook.copy_module(socket)
for op in SOCKET_OPERATIONS:
former_func = copy.deepcopy(getattr(socket, op))
setattr(modified_socket, op, generate_aikido_function(former_func, op))
setattr(socket, op, generate_aikido_function(former_func, op))

logger.debug("Wrapped `http` module")
return modified_socket
1 change: 0 additions & 1 deletion aikido_firewall/sinks/subprocess.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,5 +47,4 @@ def on_subprocess_import(subprocess):
generate_aikido_function(op=op, former_func=former_func),
)

logger.debug("Wrapped `subprocess` module")
return modified_subprocess
2 changes: 1 addition & 1 deletion aikido_firewall/sources/flask.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@
context1.set_as_current_context()
request_handler(stage="init")
except Exception as e:
logger.info("Exception on aikido __call__ function : %s", e)
logger.debug("Exception on aikido __call__ function : %s", e)

Check warning on line 63 in aikido_firewall/sources/flask.py

View check run for this annotation

Codecov / codecov/patch

aikido_firewall/sources/flask.py#L63

Added line #L63 was not covered by tests
res = flask_app.wsgi_app(environ, start_response)
return res

Expand Down
2 changes: 1 addition & 1 deletion aikido_firewall/sources/functions/request_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
context = get_current_context()
comms = get_comms()
if not context or not comms:
logger.info("Request was not complete, not running any pre_response code")
logger.debug("Request was not complete, not running any pre_response code")

Check warning on line 37 in aikido_firewall/sources/functions/request_handler.py

View check run for this annotation

Codecov / codecov/patch

aikido_firewall/sources/functions/request_handler.py#L37

Added line #L37 was not covered by tests
return

# IP Allowlist:
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[tool.poetry]
name = "aikido_firewall"
version = "0.0.9"
description = "Aikido RASP for Python"
description = "Aikido Zen for Python"
authors = ["Aikido"]
readme = "README.md"

Expand Down
2 changes: 1 addition & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = aikido_firewall
version = 1.0.0
author = Aikido
author_email = [email protected]
description = Aikido RASP for Python
description = Aikido Zen for Python
long_description = file: README.md
long_description_content_type = text/markdown
url = https://github.com/AikidoSec/firewall-python
Expand Down
Loading