Archive for the ‘security’ Category

Build/Maintain your own golden container base images

August 3, 2022 Leave a comment

Containers have become essential in the optimization of software delivery these days for any business. They can support the principals of least privilege and least access by removing most of the attack surface associated with exposing services for public consumption. They are the smallest unit that make up the 4Cs’ (CNCF uses this term to describe Cloud, Cluster, Container and Code) and have become an important part of Kubernetes management. Stripping away the complexity and isolation benefits makes them portable and it almost seems as though they have no downside right? Containers (and Kubernetes) are ephemeral and support the idea of a fully automated workload but we don’t patch them like we used to. So how do we ensure that the inevitable vulnerabilities that arise (daily if not weekly) can also be mitigated or even remediated? You start over (and over) again and again by using the ‘latest and greatest’ base images. To understand this process, we need to compare the strategy of traditional software deployment strategies and see how they differ.

First there was the base OS build where we deployed a operation System and struggled to keep it updated. We applied patches to the OS to replace any software components that needed to be replaced. Many organizations struggled with patching cadence when the fleet of systems grew to large to manage. The speed of patching needed to increase as more and more vulnerabilities were found which presented a challenge for larger organizations.

Containers start with a very small base image to provide some of the libraries that are necessary for the code that was deployed with the image. Developers need to actively minimize the components that are necessary for some core capabilities (like openssl for https, glibc for os file and device options, etc.) Failure to minimize the base image used can results in adding more and more of the libraries needed rather than relying on the benefits of a shared kernel. Best practices require the understanding of the OS being used so that the image size can be smaller and the attack surface can be reduced. This results in less vulnerabilities introduced at the container level which can result in a longer runtime using that container image.

In support of this model, it is suggested that we consider how to maintain an approved (secure) base image for any container development so our deployment strategy can make use of secure (known NOT vulnerable) images to start from. The OS manufacturers are always releasing patched versions of their base image file-systems complete with the updated components. If we consider how to turn those updated base OS images into approved secure base images, the benefits provided can increase our productivity while reducing our attack surfaces.

The process proposed here can help us obtain and build base images that have a unique hash associated with them. Since container filesystems (ausfs, overlay) can be fingerprinted, we can validate the base image hash through the entire release life-cycle. This provides an added layer of detection against rogue container use and can provide an early warning detection mechanism for both development as well as operations teams. Detecting who is using a known vulnerable base image can provide notification to be sent to application owners until those vulnerable images are removed from all of our systems.

Let me show you how this can be accomplished for any of the base images that should be approved for consumption. We start by using the ‘current build’ for any of the base OS images that we want to use. (Remember, whether your nodes run RedHat, Debian, Ubuntu, Oracle, etc. to gain the best performance and to make the best use of resources, your choice of base OS should match your node runtime version. Lets grab the latest version of the Jammy base OS for amd64 – I will use podman to build my OCI compatible image but we can also do this with docker.)

Step 1 (we should repeat this whenever there is a major change in this release. The vendor will update this daily)

root : podman import jsi-jammy-08-22

Downloading from ""
Getting image source signatures
Copying blob 911f3e234304 skipped: already exists
Copying config 4d667a55fb done
Writing manifest to image destination
Storing signatures

(Notice the hash of the base container image above. MY image was already downloaded)

Step 2 (we save the image archive now as a container to be tested)

root : podman save -o jsi-08-22.tar –format oci-archive jsi-jammy-08-22

Copying blob bb2923fbc64c done
Copying config 4d667a55fb done
Writing manifest to image destination
Storing signatures

(we are using the name of the image and the date [mm/yy] to identify it. You may also use image tags but it is best practice to use unique naming)

Step 3 (lets save some space and compress it)

root : gzip -9 jsi-jammy-08-22.tar – ( results in the image named jsi-jammy-08-22.tar.gz)

Final step is to run it though a security scan to ensure there are no high or critical vulnerabilities contained in this base image.

C:\image>snyk container test oci-archive:jsi-jammy-08-22.tar.gz

Testing oci-archive:jsi-jammy-08-22.tar.gz…

✗ Low severity vulnerability found in tar
Description: NULL Pointer Dereference
Introduced through: meta-common-packages@meta
From: meta-common-packages@meta > tar@1.34+dfsg-1build3

✗ Low severity vulnerability found in shadow/passwd
Description: Time-of-check Time-of-use (TOCTOU)
Introduced through: shadow/passwd@1:4.8.1-2ubuntu2, adduser@3.118ubuntu5, shadow/login@1:4.8.1-2ubuntu2
From: shadow/passwd@1:4.8.1-2ubuntu2
From: adduser@3.118ubuntu5 > shadow/passwd@1:4.8.1-2ubuntu2
From: shadow/login@1:4.8.1-2ubuntu2

✗ Low severity vulnerability found in pcre3/libpcre3
Description: Uncontrolled Recursion
Introduced through: pcre3/libpcre3@2:8.39-13ubuntu0.22.04.1, grep@3.7-1build1
From: pcre3/libpcre3@2:8.39-13ubuntu0.22.04.1
From: grep@3.7-1build1 > pcre3/libpcre3@2:8.39-13ubuntu0.22.04.1

✗ Low severity vulnerability found in pcre2/libpcre2-8-0
Description: Out-of-bounds Read
Introduced through: meta-common-packages@meta
From: meta-common-packages@meta > pcre2/libpcre2-8-0@10.39-3build1

✗ Low severity vulnerability found in pcre2/libpcre2-8-0
Description: Out-of-bounds Read
Introduced through: meta-common-packages@meta
From: meta-common-packages@meta > pcre2/libpcre2-8-0@10.39-3build1

✗ Low severity vulnerability found in ncurses/libtinfo6
Description: Out-of-bounds Read
Introduced through: ncurses/libtinfo6@6.3-2, bash@5.1-6ubuntu1, ncurses/libncurses6@6.3-2, ncurses/libncursesw6@6.3-2, ncurses/ncurses-bin@6.3-2, procps@2:3.3.17-6ubuntu2, util-linux@2.37.2-4ubuntu3, ncurses/ncurses-base@6.3-2
From: ncurses/libtinfo6@6.3-2
From: bash@5.1-6ubuntu1 > ncurses/libtinfo6@6.3-2
From: ncurses/libncurses6@6.3-2 > ncurses/libtinfo6@6.3-2
and 10 more...

✗ Low severity vulnerability found in krb5/libkrb5support0
Description: Integer Overflow or Wraparound
Introduced through: krb5/libkrb5support0@1.19.2-2, adduser@3.118ubuntu5, krb5/libk5crypto3@1.19.2-2, krb5/libkrb5-3@1.19.2-2, krb5/libgssapi-krb5-2@1.19.2-2
From: krb5/libkrb5support0@1.19.2-2
From: adduser@3.118ubuntu5 > shadow/passwd@1:4.8.1-2ubuntu2 > pam/libpam-modules@1.4.0-11ubuntu2 > libnsl/libnsl2@1.3.0-2build2 > libtirpc/libtirpc3@1.3.2-2ubuntu0.1 > krb5/libgssapi-krb5-2@1.19.2-2 > krb5/libkrb5support0@1.19.2-2
From: adduser@3.118ubuntu5 > shadow/passwd@1:4.8.1-2ubuntu2 > pam/libpam-modules@1.4.0-11ubuntu2 > libnsl/libnsl2@1.3.0-2build2 > libtirpc/libtirpc3@1.3.2-2ubuntu0.1 > krb5/libgssapi-krb5-2@1.19.2-2 > krb5/libk5crypto3@1.19.2-2 > krb5/libkrb5support0@1.19.2-2
and 8 more...

✗ Low severity vulnerability found in gmp/libgmp10
Description: Integer Overflow or Wraparound
Introduced through: gmp/libgmp10@2:6.2.1+dfsg-3ubuntu1, coreutils@8.32-4.1ubuntu1, apt@2.4.6
From: gmp/libgmp10@2:6.2.1+dfsg-3ubuntu1
From: coreutils@8.32-4.1ubuntu1 > gmp/libgmp10@2:6.2.1+dfsg-3ubuntu1
From: apt@2.4.6 > gnutls28/libgnutls30@3.7.3-4ubuntu1 > gmp/libgmp10@2:6.2.1+dfsg-3ubuntu1
and 1 more...

✗ Low severity vulnerability found in glibc/libc-bin
Description: Allocation of Resources Without Limits or Throttling
Introduced through: glibc/libc-bin@2.35-0ubuntu3.1, meta-common-packages@meta
From: glibc/libc-bin@2.35-0ubuntu3.1
From: meta-common-packages@meta > glibc/libc6@2.35-0ubuntu3.1

✗ Low severity vulnerability found in coreutils
Description: Improper Input Validation
Introduced through: coreutils@8.32-4.1ubuntu1
From: coreutils@8.32-4.1ubuntu1

✗ Medium severity vulnerability found in perl/perl-base
Description: Improper Verification of Cryptographic Signature
Introduced through: meta-common-packages@meta
From: meta-common-packages@meta > perl/perl-base@5.34.0-3ubuntu1
Tested 102 dependencies for known issues, found 11 issues.


(Look Ma, no high or critical findings!)

Now we have a base OS image ready to be used with any new/existing container build process. Best practices include the ability to digitally sign these images so that build pipelines can verify that any images being included are tested and approved. We can remove the previous version of the base OS image and provide a notice to current/future users that vulnerabilities have been found in the previous version. Dev teams can bump the version in any code they have and begin to test if there are any breaking changes that would require refactoring. Even if there is no change in the code, they must release their containers using these new base OS images to mitigate any vulnerabilities that are introduced.

Categories: security, Work related Tags: ,

After the breach…

July 26, 2022 Leave a comment

Accidents happen and in the security field, they are usually called an ‘0-day’.

There are (at least) three questions you may be asked by your board, about your AppSec program…

  • Was all the software tested using all of our controls & capabilities that were applicable?
  • Did all the findings that were produced measure below our acceptable risk ratings?
  • Were any/all of the vulnerabilities being fixed according to our accepted remediation timelines?

Lets unpack that for everyone in an attempt to understand the motivations of some of our brightest ‘captains’. (If I was a board member…)

Misinformation – Does this event signal a lack of efficacy of our overall Appsec program? Do the controls work according to known practices? Perhaps, this is an anomaly, an edge case that now requires additional investment? What guarantees do we have that any correction strategy will be effective? If changes are warranted, which part should we focus on, People, Process or Technology?

Jeff says – changing the program can take a large investment for any/all of these. Get back to the basics and start with some metrics to see if you have effective coverage first. Prioritize making policy/configuration visible for each implementation of your security tools and aim for all of your results in one tool.

Liability – Is our security assessment program effective enough? Does this blind spot show us the inability to understand/avoid these threats at scale? Does this event indicate a systemic failure to detect/prevent this type of threat in the future?

Jeff says – Push results from Pentesting/Red Team/Security Ops back into the threat model and show if/how any improvement can be effective. Moving at the speed of DevOps means running more tests, more often, and correlating the findings to show value through velocity by catching and fixing them quickly.

Profit and Loss – Do we have a software quality problem that may require us to consider an alternative resource pool? If digitization is increasing in cost due to loss, maybe we need to improve our control capabilities to detect/prevent bad software from reaching production? Maybe we should take additional steps to ensure we have the right development teams to avoid mistakes?

Jeff says – to stop the bleeding, you might consider a different source of secure code? You might also consider an adjustment to your secure training programs? Maybe your security analysts are having their own quality issues? Consider raising the threshold of approved tools to be considered? Broker communication for your dev teams to take on more of the security responsibility.

For any leadership who is dealing with CyberSecurity these days, these are all very good questions. Security is Hard, Application Security, Cloud Security, Data Security – they are ALL hard individually so how does any one person/team understand them entirely?

I began to ask myself that question almost a decade ago during my mobile penetration testing period. When Facebook had created React which involved more than one software language in the same project. I found a cross site scripting flaw in the mobile client during testing which I felt pretty confident was NOT a false positive. I decided to check the static code findings to see if this could be correlated. (We can save the rest of that story for another blog post).

A light went off in my head, ‘correlation between two or more security tools in a single pane of glass’. What an idea – you need something that can pull in all of the datasets (finding reports) and provide some deduplication (so we don’t give dev teams multiple findings from multiple tools), just the fact that we are confident of the viability of the finding. I investigated some of the tool vendors and worked with them for a few years while the capability began to mature in the industry.

Today, Gartner calls this space Application Security Orchestration and Correlation, a contraction of ‘security orchestration’ (where you apply policy as code) and correlating, deduping the results. When done successfully, it also provides a single pane of glass for the operations team or any other orchestration or reporting software in use in your org. Think of it as the one endpoint with all the answers; a way to abstract away the API schema and various life-cycle changes that are associated with new and existing tool-sets.

Whether you wish to interconnect all of your existing orchestration tooling for your pipelines & other infrastructure or perhaps you want to build out your security governance capabilities by conducting all of your own security testing, ASOC tools are capable of providing security at the speed of DevOps.

There really is no other way to accomplish it at scale!

Categories: security, Work related Tags: ,

Infosec not your job but your responsibility? How to be smarter than the average bear

July 25, 2022 Leave a comment

Want to measure how beneficial it is for your software development teams to learn to think more like an adversary? Just look at the first 20 years of use against the last 10-20?

Cloud Vulnerabilities & Security Issues Database

July 22, 2022 Leave a comment

For those of us who thought the rising list of new CVEs was bad enough, now comes a new list of cloud platform relayed vulnerabilities that only your choice of CSP is responsible to fix.

Time to update your threat models folks!

Categories: security Tags: ,

SLSA • Supply-chain Levels for Software Artifacts

Oh thank God I am not the only one who sees the next techopolus (O-day apocalypse) as we all adopt kubernetes as the orchestration platform and we forget about *where* those container images come from…

Categories: security Tags: , , ,

Bank API as Microservices with CQRS in TypeScript | Level Up Coding

Very seldom, do we get to see several technologies used so well together. With the exception of how to illustrate how the secrets should be managed, this article really shows what secure by design is all about.

Categories: security Tags: ,

New European rules for mobile banking apps coming to a device near you…

July 26, 2019 Leave a comment

The world is clearly a better place now that we carry computers in our back pocket but we need an increase in security measures for payment transactions and therefore we will require an increase in regulation, such as the PSD2 from European Commission.

The Payment Services Directive mandates compliance by September 2019 and aims to regulate banks, payment service providers and electronic payments to include security features to protect consumers across digital channels. The PSD2 legislation will require financial services in the European Union (EU) to contribute to a more integrated, secure, and efficient payments ecosystem.

The PSD2 directive requires financial institutions to:

  • Provide/Implement a monitoring mechanism in their apps to detect/report signs of malware.
  • Provide security measures in their app to mitigate risk for the user device.
  • Ensure consumers have a secure environment to execute their financial transactions

In Article 2 and Article 9 of the directive, PSD2 highlights Strong Customer Authentication (SCA) and Safe Execution Environment (SEE), which requires de-risking across various threat vectors impacting mobile apps.

These include detecting compromised devices (eg: jailbroken or rooted), unsafe environments (such as a fake or malicious wi-fi), as well as malware and vulnerabilities within the application execution environment. PSD2 also includes RTS (Regulatory Technical Standards), which are regulatory requirements set by the European Banking Authority (EBA) to ensure that payments across the EU are secure, fair & efficient.

To meet these requirements, financial institutions should add strong security capabilities like binary protections to their mobile apps. These controls are designed to protect against known and unknown threats on users’ devices.

Mobile banking apps should also be able to detect when they are installed on risky devices and consider restricting access to high value banking services until those risks have been remediated.

Categories: Mobile, security Tags: ,

Web servers are still vulnerable…

April 28, 2018 Leave a comment

In a survey published on an often referenced support site for developers (Stack Overflow), they recently confirmed that JavaScript is the most popular programming language for the 6th year in a row. Almost 70% of the respondents claim that they visit searching for help on this subject so it may not come as a surprise that JavaScript is also the primary cause of vulnerabilities on websites today.

In a blog post from the vendor that brings us one of the most popular tool for hacking websites and finding vulnerabilities, Portswigger writes a great article in which they detail a number of methods that can be used to abuse JavaScript and to bypass cross site scripting mitigation by most frameworks.

There are thousands of ways that can be used to bypass XSS in websites and web developers should already know this. XSS is the number one method to compromise a browser which, in combination with privilege escalation can allow an attacker to take over your computer. Even script kiddies can capture session tokens or cookies from websites without proper security controls that can be used to login as you without even knowing your password. Here is a list of the risks in order of importance for an attacker;

  1. Account hijacking
  2. Credential stealing
  3. Sensitive Data Leakage
  4. Drive by Downloading
  5. Keyloggers/Scanners
  6. Vandalism

Don’t ignore these risks on your websites, public facing or not. If you login to a website often in your organization and it is vulnerable to cross site scripting, teach your users how to identify security risks that could be used to harvest credentials and expose them to malicious attacks. You may also want to make sure that your sites are tested to ensure they are not vulnerable to this type of attack. With Phishing attacks being the number one method that pentesters gain access to your organization, xss is the primary method being used.


Categories: security, Work related Tags: ,

RFID hacking for fun (and profit)

November 11, 2016 1 comment

I recently received a new proxmark3 easy and began the fun of reader and cloning access badge cards. For those of you unaware of how these little while cards work, allow me to share some fun facts. The Radio Frequency Identification Device (RFID) Card are comprised of a coil of copper wire wrapped in a loop to create an electronic field. They also have an Integrated circuit (IC) embedded in them that is powered when the field is oscillated. Remember how rubbing your hands together creates static electricity?proxcard

This oscillating field comes from the reader and is tuned to a specific frequency that can be used to pickup a unique identifier from your card.

Near Field Communication (NFC) uses this mechanism and has a more dense command set which can allow it to utilize encryption.

We use RFID tags on most consumer goods to prevent theft and there was once an idea to embed these into pets and even humans!

proxmark3With a little computer know-how  and about $100.00 you can create a device that can be used to clone these badges in just seconds.

I setup my device to capture some data from my work facility with relative ease. Once stored (captured) from any card, the buffer of the device can now replay the signal to fool any reader.

I wanted to show you what some of the thinest cards would look like when they are scanned. The version I am using is a 37 bit iClass px D8L and it features the facility code and the card holder number (blanked out for security purposes).


Verifying the data from any card is as simple as issuing the command ‘lf search’. Here we can see the card number (known as the TAG ID) that would be registered into the security system along with the format length.

Now for the fun part – we place our valid card on top of our reader and issue the following command – ‘lf hid fskdemod’. This will tell you the TAG ID and will repeat quite a few times until it has sampled the modulated waveform.

Now we place a T5577 card on our proxmark device and type ‘lf hid clone’ followed by the TAG ID number. With any luck, you now have a cloned copy of your card!

Beware of manned security stations using newer technology, they will often look at the face of anyone who had their picture taken while being issued a card. Unless you look very similar to the person whose card you have cloned, you will surely be caught.

Next step is to setup a malware program on the security computer so that your picture is substituted for the card holder and our physical security challenge is successful. I wonder what their favourite website is to use when there is no one coming in and out of the building….hmmm. Nextime.


Categories: security Tags: ,

Playing with ASLR for Linux

November 5, 2016 Leave a comment

I recently completed a certification with SANS where we were taught to create our own exploits. A well behaved program should have a start procedure called a prologue, clean up and maintain itself and any objects it creates and finally an end. During our testing we assumed that Address Space Layout Randomization (ASLR) was turned on in order for our exploits to be successful but this is not always the case.

All programs have two different sections (instructions and data) and they need to be flagged differently in the memory. A section where only normal data is stored should be marked as non-executable. Executable code that does not dynamically change, should be flagged as read-only.

One of the tricks that hackers use to get control is to hijacking the stack pointer (a road map for where to go next). Evil programs are designed to perform a redirection in order to run their own malicious code. When any program is running in memory, if we want to protect it against evil programs we can use ASLR. Address space layout randomization is based upon the low chance of an attacker guessing the locations of randomly placed areas. Security is increased by increasing the search space.


One of the easiest ways to find if a Linux server had ASLR turned on was to look at the proc filesystem for the variable ‘randomize_va_space’.

cat /proc/sys/kernel/randomize_va_space

If the value returned was a zero (0) then it was off, a one (1) then it was on for the stack. Ideally you want to use the value two (2) which will also randomize the data segments but all programs need to be built as a Position Independent Executable (PIE). It will also require that all shared object libraries (.so libraries) also be built as PIE.

Another way to see if ASLR has been activated is to run ldd against your executable and inspect the load address of the shared object libraries.

$ ldd -d -r demo => (0x00007fffe2fff000) => /lib64/ (0x00007f3e73cd8000)
/lib64/ (0x00007f3e74057000)
$ ldd -d -r demo => (0x00007fff053ff000) => /lib64/ (0x00007f82071ac000)
/lib64/ (0x00007f820752b000)

Notice the change in the memory address referenced by the C library and the dynamic linker (libc and ld-Linux-x86-64)? This indicates that ASLR is in effect.

To ensure that all of your Linux code is more difficult to bypass you will want to audit your systems and look for the value of 2 (assuming your other programs fully support PIE.

root@mybox:~$ sysctl -a –pattern “randomize”

kernel.randomize_va_space = 2


I refer to ‘Bypassing ASLR’ on the Linux system as finding a way to defeat it as a security mechanism. If all shared libraries were compiled as PIE along with all executables (services if attacking remotely and all executables if granted console access) then ASLR would be very difficult to beat but that is not the case. The additional checks required to be sure that the attack footprint has been reduced to zero is to audit each of the services publically available to make sure that all libraries and executables involved are 100% position independent.

There is a great opensource tool out there for checking if your executables support PIE.

‘’ maintained by slimm609 (

Categories: security, Work related