Hello,

I'm trying to find out if there are any plans for "the original" scapy to support Python 3 or not.

The only references to real code I found is in https://github.com/secdev/scapy/issues/87#issuecomment-258094860 but it seems to me that the plan for scapy rewrite is not moving forward because repo https://bitbucket.org/secdev/scapy3-prototype2 has last commit from 2015-02-25.

Given that current Fedora (and so future of RHEL/CentOS/Scientific Linux) is migrating away from Python 2 (as well as all other Linux distros) this forces me to ask the question again: What are the plans for Python 3 support in scapy?

Without some form plan, Fedora/RHEL/CentOS will have to implement the same approach as Debian (see https://github.com/secdev/scapy/issues/87#issuecomment-257431044), i.e. use scapy3k instead of scapy.

If this is the case, are there any plans to start merging features between scapy and scapy3k?

Thank you for your time and answers.

This PR speeds up windows booting (by a lot on windows 7):

• pcapdnet cache
• remove useless function
• do not read routes twice in get_working_if
• fixes route_add_loopback
• re-uses the same powershell instance while booting by adding a Powershell Manager
• fix long IP lists (especially IPv6 possible IPs): do not crop
• minor bug fixes (python3)

Results:

>>> a = time.time()
...:for i in range(1,10):
...:    read_routes[6]()
...:print(time.time()-a)

on Windows 10:

| | Normal | After POWERSHELL_PROCESS.__init__() (=during of after booting) | | --|---|---| | 10 x read_routes | 8.33999991417s | 0.571000051498s | | 10 x read_routes6 | 9.13199996948s | 0.917000055313s | | 10 x get_windows_if_list | 12.8439998627s | 1.46499991417s |

POWERSHELL_PROCESS auto-turns off when scapy has finished booting (this could be changed, but do we want to leave a background task powershell open ?)

Considered on its own, the implementation of the IKEv2 protocol is quite mature, however the integration with the other protocols, in particular with the closely related protocols ISAKMP (= IKEv1, its predecessor) and ESP is unfinished and slightly broken.

The fact that the IKEv2 support needs to be loaded manually is only a minor flaw compared to the fact that the module essentially only replaces ISAKMP bindings with IKEv2 bindings and does not solve the problem of coexistence of the three protocols.

In particular, Scapy's dissector is currently missing the the following features:

• automatic switching between IKEv2 and ISAKMP based on the protocol version
• UDP Encapsulation of ESP and IKE packets on port 4500 for NAT traversal

As a consequence, packet captures of VPN traffic are not dissected correctly.

This commit fixes the problem by implementing the missing features, Moving the IKEv2 implementation from 'scapy.contrib' to 'scapy.layer' was a natural choice which not only simplified the fix but is also justified by the fact that this fix turns IKEv2 is a full-featured Scapy layer which simply works out-of-the-box.

Checklist:

• [x] I have checked CONTRIBUTING.md
• [x] I squashed commits belonging together
• [x] I added unit tests or explained why they are not relevant
• [x] I added example packet captures
• [x] I executed the regression tests (using cd test && ./run_tests or tox)

Check this branch for detailed history of this change: https://github.com/phretor/scapy/commits/rtps

Disclaimer: I'm not sure this is the right way to bind layers that are not restricted to a non-fixed port range, so I put a comment. Let me or @vmayoral know if you prefer to enable the "ugly hack". All dissection tests pass even now.

# layer binding
# for port in range(7400, 7500):  # very ugly hack :-)
#     bind_layers(UDP, RTPS, dport=port)
#     bind_layers(TCP, RTPS, dport=port)

bind_layers(UDP, RTPS)
bind_layers(RTPS, RTPSMessage, magic=b"RTPS")
bind_layers(RTPS, RTPSMessage, magic=b"RTPX")

Co-authored-by: Federico Maggi [email protected] Co-authored-by: Víctor Mayoral Vilches [email protected] Signed-off-by: Federico Maggi [email protected]

@KeksMassacre The following to GMLAN Utils-Tests aren't working with ISOTPSoftSockets after a refactoring of ISOTPUnitTests #2341

This is a reminder to fix this unit tests

https://github.com/secdev/scapy/blob/f1e632b9a78029578371d7b27f2f116945da1049/test/contrib/automotive/gm/gmlanutils.uts#L490-L491 https://github.com/secdev/scapy/blob/f1e632b9a78029578371d7b27f2f116945da1049/test/contrib/automotive/gm/gmlanutils.uts#L880-L881

This is just a PoC. A *.uts file will be transformed to a pytest like file. The exported file usually needs manual modifications, but this exporter can be a first step towards pytest

This avoids duplicated code and brings to Windows enhancements that exist for Unix (sniff() on multiple interfaces, parallel sndrcv()).

It also brings a huge code clean-up on both functions and may have a (limited but positive) impact on performances (some tests have been moved outside of sniff loops). The code is (IMO) much more readable and maintainable.

It changes the behavior of sniff(): it is now possible to provide lists for offline and opened_socket parameters (that was only possible for iface). It is also possible, for those three parameters, to provide dict objects mapping an element (interface or file name, or opened socket) to a label.

Also, fixes #780.

@TabeaSpahn @polybassa

We would be interested in the opinions of the maintainers. It took some time but we think it's finally stable enough to be shown in a PR.

Python 2 and Python 3 supported.

Things that probably have to be changed:

• [x] Merge tests
• [x] Add urwid as an optional dependency to scapy and maybe output a nice error message if the packet viewer is imported even though urwid is not installed
• [x] Add urwid to Travis server
• [x] Fix flake8 errors
• [x] Remove debugging scripts
• [x] Improve documentation (for example add a screenshot of the viewer)
• [x] Show time relative to first packet

Requirements: pip3 install urwid

Cool feature I want to highlight: It's a TUI working fully over ssh which means also mouse interaction is possible over ssh.

EDIT: That PR was splitted in smaller ones. This will stay open for legacy until all have been merged

Next part of supporting Python 3. This increases the compatibility between python 2 and 3, by using common methods + six.py module. It does not provide a suitable python 3 compatibility yet.

The PR:

• used Python-modernize as a base to update scapy's code
• manually fixed some conflicts/bugs
• Included six.py in modules
• fix some small bugs (windows)

Well this is https://github.com/secdev/scapy/pull/380 and https://github.com/secdev/scapy/pull/397 but a way more cleaner...

IT IS FINALLY WORKING

So as today, IPv6 routes are not supported by scapy on Windows. Here is the IPv6 routes support on Windows 😃

This patch:

• Fixes some route6.py functions to work on Windows
• Removes double warning logs when loading scapy (when winpcap is not installed)
• Add regression.uts tests

What it returns: http://pastebin.com/hUU6ykcc

I've tested it: it's possible to send IPv6 packets

Hello!

I made a few fixes and enhancements in the UTscapy file:

• Fix trailing whitespace, unused variable and unused imports.
• Protect file opening in 'with' statements
• Use local version of UTscapy.js and UTscapy.css
• Allow continuing tests after a failed campaign. Add an option to keep old behavior

And the main feature is to be able to interrupt a serie of tests using Control-C.

3GPP TS 29.274 states in Section 5.5.1: "Octets 3 to 4 represent the Message Length field. This field shall indicate the length of the message in octets excluding the mandatory part of the GTP-C header (the first 4 octets). The TEID (if present) and the Sequence Number shall be included in the length count."

Checklist:

• [x] If you are new to Scapy: I have checked CONTRIBUTING.md (esp. section submitting-pull-requests)
• [ ] I squashed commits belonging together
• [x] I added unit tests or explained why they are not relevant
• [x] I executed the regression tests (using cd test && ./run_tests or tox)
• [ ] If the PR is still not finished, please create a Draft Pull Request

Brief description

In the current scapy implementation of Netflow protocol, it seems like the Options-Templates Flowset scapy.layers.netflow.NetflowOptionsFlowsetV9 length field does not include the padding length.

However, Section 6.1 of RFC 3954 (NetflowV9), states that

Padding The Exporter SHOULD insert some padding bytes so that the subsequent FlowSet starts at a 4-byte aligned boundary. It is important to note that the Length field includes the padding bytes. Padding SHOULD be using zeros.

Is it a misunderstanding of my side, or is it really a bug ?

Thanks !

Scapy version

2.4.5

Python version

3.10

Operating system

Linux 5.15.0

Additional environment information

No response

How to reproduce

from scapy.layers.netflow import NetflowHeader, NetflowHeaderV9, NetflowOptionsFlowsetOptionV9, NetflowOptionsFlowsetV9, NetflowOptionsFlowsetScopeV9, NetflowDataflowsetV9

templateFlowSet_256 = NetflowOptionsFlowsetV9(
	templateID = 256,
	option_scope_length = 4*1,
	option_field_length = 4*3,
	scopes = [
		NetflowOptionsFlowsetScopeV9(scopeFieldType=1,	scopeFieldlength= 4),
	],
	options = [
		NetflowOptionsFlowsetOptionV9(optionFieldType= 10,	optionFieldlength= 4),
		NetflowOptionsFlowsetOptionV9(optionFieldType= 82,	optionFieldlength= 32),
		NetflowOptionsFlowsetOptionV9(optionFieldType= 83,	optionFieldlength= 240)
	])

netflow_header = NetflowHeader() / NetflowHeaderV9()


pkt = netflow_header / templateFlowSet_256

pkt.show2()

Actual result

###[ Netflow Header ]### 
  version   = 9
###[ Netflow Header V9 ]### 
     count     = 1
     sysUptime = 0
     unixSecs  = Thu, 01 Jan 1970 00:00:00 +0000 (0)
     packageSequence= 0
     SourceID  = 0
###[ Netflow Options Template FlowSet V9 ]### 
        flowSetID = 1
        length    = 26
        templateID= 256
        option_scope_length= 4
        option_field_length= 12
        \scopes    \
         |###[ Netflow Options Template FlowSet V9/10 - Scope ]### 
         |  scopeFieldType= System
         |  scopeFieldlength= 4
        \options   \
         |###[ Netflow Options Template FlowSet V9/10 - Option ]### 
         |  enterpriseBit= 0
         |  optionFieldType= INPUT_SNMP
         |  optionFieldlength= 4
         |###[ Netflow Options Template FlowSet V9/10 - Option ]### 
         |  enterpriseBit= 0
         |  optionFieldType= IF_NAME
         |  optionFieldlength= 32
         |###[ Netflow Options Template FlowSet V9/10 - Option ]### 
         |  enterpriseBit= 0
         |  optionFieldType= IF_DESC
         |  optionFieldlength= 240
        pad       = ''
###[ Netflow FlowSet V9/10 ]### 
           flowSetID = 0
           length    = None
           \templates \

Expected result

• pre_build length is : 10 (flowsetID + length + templateID) + 4 (scope) + 12 (options) = 26 bytes
• padding should be of 26 % 4 = 2 bytes
• post_build Flowset length after post_build() should be of 26 + 2 (padding) = 28 bytes

###[ Netflow Header ]### 
  version   = 9
###[ Netflow Header V9 ]### 
     count     = 1
     sysUptime = 0
     unixSecs  = Thu, 01 Jan 1970 00:00:00 +0000 (0)
     packageSequence= 0
     SourceID  = 0
###[ Netflow Options Template FlowSet V9 ]### 
        flowSetID = 1
        length    = 28
        templateID= 256
        option_scope_length= 4
        option_field_length= 12
        \scopes    \
         |###[ Netflow Options Template FlowSet V9/10 - Scope ]### 
         |  scopeFieldType= System
         |  scopeFieldlength= 4
        \options   \
         |###[ Netflow Options Template FlowSet V9/10 - Option ]### 
         |  enterpriseBit= 0
         |  optionFieldType= INPUT_SNMP
         |  optionFieldlength= 4
         |###[ Netflow Options Template FlowSet V9/10 - Option ]### 
         |  enterpriseBit= 0
         |  optionFieldType= IF_NAME
         |  optionFieldlength= 32
         |###[ Netflow Options Template FlowSet V9/10 - Option ]### 
         |  enterpriseBit= 0
         |  optionFieldType= IF_DESC
         |  optionFieldlength= 240
        pad       = '\x00\x00'

Related resources

• RFC 3954, see section 6.1, p. 15
• Fix proposition if the issue is confirmed to be a bug

Brief description

See this cange to ber.py of PR #3693:

- s = BER_id_enc(implicit_tag) + s[1:]
+ s = BER_id_enc((hash(hidden_tag) & ~(0x1f)) | implicit_tag) + s[1:]

hash(hidden_tag) will resolve to the enum value of custom tags. When OR'ed together with an implicit tag, nothing good happens.

As a side note, if hidden_tag ever was None: hash(None) is a large integer, yielding even more garbage.

Scapy version

2.5.0

Python version

3.8.12

Operating system

RH8

Additional environment information

No response

How to reproduce

from scapy.asn1.asn1 import ASN1_Codecs
from scapy.asn1packet import ASN1_Packet
from scapy.layers.ldap import ASN1F_LDAP_Authentication_simple
 
class TestAuth(ASN1_Packet):
    ASN1_codec = ASN1_Codecs.BER
    ASN1_root = ASN1F_LDAP_Authentication_simple("auth", None, implicit_tag=129)
 
test = TestAuth()
test.auth = b'abc'
print(bytes(test).hex())

Actual result

a103616263

Expected result

8103616263

Related resources

No response

Adds the Control PDUs that were added in spec revision 5.1, 5.2 and 5.3.

This changes only support using a scapy as a packet builder. Dissection is not yet implemented.

This PR will add 802.11 action frame categories (802.11-2016 9.4.1.11), 802.11 WNM frame action fields (802.11-2016 9.6.14.1), and 802.11v BSS transition (BTM) request and response frames (802.11-2016 9.6.14.9-10) to the dot11 layer.

802.11v BTM frames are part of the WNM protocol, which is a category of action management frames. They are used to steer wireless stations to and from a BSS. This PR includes required fields and optional Neighbor Report element field. Neighbor Report (802.11-2016 9.4.2.37) element can contain various sub elements; only BSS Termination Duration sub element is added, and for the rest, generic SubelemTLV is used.

Tests: air-sniffed BTM request and response frames are added. For response, a frame with status code = 6 is used, which contains the Neighbor Report element and will provide more coverage for the test.