You are viewing the version of this documentation from Perl 5.39.2. This is a development version of Perl.

CONTENTS

NAME

perl5320delta - what is new for perl v5.32.0

DESCRIPTION

This document describes differences between the 5.30.0 release and the 5.32.0 release.

If you are upgrading from an earlier release such as 5.28.0, first read perl5300delta, which describes differences between 5.28.0 and 5.30.0.

Core Enhancements

The isa Operator

A new experimental infix operator called isa tests whether a given object is an instance of a given class or a class derived from it:

if( $obj isa Package::Name ) { ... }

For more detail see "Class Instance Operator" in perlop.

Unicode 13.0 is supported

See https://www.unicode.org/versions/Unicode13.0.0/ for details.

Chained comparisons capability

Some comparison operators, as their associativity, chain with some operators of the same precedence (but never with operators of different precedence).

if ( $x < $y <= $z ) {...}

behaves exactly like:

if ( $x < $y && $y <= $z ) {...}

(assuming that "$y" is as simple a scalar as it looks.)

You can read more about this in perlop under "Operator Precedence and Associativity" in perlop.

New Unicode properties Identifier_Status and Identifier_Type supported

Unicode has revised its regular expression requirements: https://www.unicode.org/reports/tr18/tr18-21.html. As part of that they are wanting more properties to be exposed, ones that aren't part of the strict UCD (Unicode character database). These two are used for examining inputs for security purposes. Details on their usage is at https://www.unicode.org/reports/tr39/.

It is now possible to write qr/\p{Name=...}/, or qr!\p{na=/(SMILING|GRINNING) FACE/}!

The Unicode Name property is now accessible in regular expression patterns, as an alternative to \N{...}. A comparison of the two methods is given in "Comparison of \N{...} and \p{name=...}" in perlunicode.

The second example above shows that wildcard subpatterns are also usable in this property. See "Wildcards in Property Values" in perlunicode.

Improvement of POSIX::mblen(), mbtowc, and wctomb

The POSIX::mblen(), mbtowc, and wctomb functions now work on shift state locales and are thread-safe on C99 and above compilers when executed on a platform that has locale thread-safety; the length parameters are now optional.

These functions are always executed under the current C language locale. (See perllocale.) Most locales are stateless, but a few, notably the very rarely encountered ISO 2022, maintain a state between calls to these functions. Previously the state was cleared on every call, but now the state is not reset unless the appropriate parameter is undef.

On threaded perls, the C99 functions mbrlen(3), mbrtowc(3), and wcrtomb(3), when available, are substituted for the plain functions. This makes these functions thread-safe when executing on a locale thread-safe platform.

The string length parameters in mblen and mbtowc are now optional; useful only if you wish to restrict the length parsed in the source string to less than the actual length.

Alpha assertions are no longer experimental

See "(*pla:pattern)" in perlre, "(*plb:pattern)" in perlre, "(*nla:pattern)" in perlre>, and "(*nlb:pattern)" in perlre. Use of these no longer generates a warning; existing code that disables the warning category experimental::alpha_assertions will continue to work without any changes needed. Enabling the category has no effect.

Script runs are no longer experimental

See "Script Runs" in perlre. Use of these no longer generates a warning; existing code that disables the warning category experimental::script_run will continue to work without any changes needed. Enabling the category has no effect.

Feature checks are now faster

Previously feature checks in the parser required a hash lookup when features were set outside of a feature bundle, this has been optimized to a bit mask check. [GH #17229]

Perl is now developed on GitHub

Perl is now developed on GitHub. You can find us at https://github.com/Perl/perl5.

Non-security bugs should now be reported via GitHub. Security issues should continue to be reported as documented in perlsec.

Compiled patterns can now be dumped before optimization

This is primarily useful for tracking down bugs in the regular expression compiler. This dump happens on -DDEBUGGING perls, if you specify -Drv on the command line; or on any perl if the pattern is compiled within the scope of use re qw(Debug DUMP_PRE_OPTIMIZE) or use re qw(Debug COMPILE EXTRA). (All but the second case display other information as well.)

Security

[CVE-2020-10543] Buffer overflow caused by a crafted regular expression

A signed size_t integer overflow in the storage space calculations for nested regular expression quantifiers could cause a heap buffer overflow in Perl's regular expression compiler that overwrites memory allocated after the regular expression storage space with attacker supplied data.

The target system needs a sufficient amount of memory to allocate partial expansions of the nested quantifiers prior to the overflow occurring. This requirement is unlikely to be met on 64-bit systems.

Discovered by: ManhND of The Tarantula Team, VinCSS (a member of Vingroup).

[CVE-2020-10878] Integer overflow via malformed bytecode produced by a crafted regular expression

Integer overflows in the calculation of offsets between instructions for the regular expression engine could cause corruption of the intermediate language state of a compiled regular expression. An attacker could abuse this behaviour to insert instructions into the compiled form of a Perl regular expression.

Discovered by: Hugo van der Sanden and Slaven Rezic.

[CVE-2020-12723] Buffer overflow caused by a crafted regular expression

Recursive calls to S_study_chunk() by Perl's regular expression compiler to optimize the intermediate language representation of a regular expression could cause corruption of the intermediate language state of a compiled regular expression.

Discovered by: Sergey Aleynikov.

Additional Note

An application written in Perl would only be vulnerable to any of the above flaws if it evaluates regular expressions supplied by the attacker. Evaluating regular expressions in this fashion is known to be dangerous since the regular expression engine does not protect against denial of service attacks in this usage scenario.

Incompatible Changes

Certain pattern matching features are now prohibited in compiling Unicode property value wildcard subpatterns

These few features are either inappropriate or interfere with the algorithm used to accomplish this task. The complete list is in "Wildcards in Property Values" in perlunicode.

Unused functions POSIX::mbstowcs and POSIX::wcstombs are removed

These functions could never have worked due to a defective interface specification. There is clearly no demand for them, given that no one has ever complained in the many years the functions were claimed to be available, hence so-called "support" for them is now dropped.

A bug fix for (?[...]) may have caused some patterns to no longer compile

See "Selected Bug Fixes". The heuristics previously used may have let some constructs compile (perhaps not with the programmer's intended effect) that should have been errors. None are known, but it is possible that some erroneous constructs no longer compile.

\p{user-defined} properties now always override official Unicode ones

Previously, if and only if a user-defined property was declared prior to the compilation of the regular expression pattern that contains it, its definition was used instead of any official Unicode property with the same name. Now, it always overrides the official property. This change could break existing code that relied (likely unwittingly) on the previous behavior. Without this fix, if Unicode released a new version with a new property that happens to have the same name as the one you had long been using, your program would break when you upgraded to a perl that used that new Unicode version. See "User-Defined Character Properties" in perlunicode. [GH #17205]

Modifiable variables are no longer permitted in constants

Code like:

my $var;
$sub = sub () { $var };

where $var is referenced elsewhere in some sort of modifiable context now produces an exception when the sub is defined.

This error can be avoided by adding a return to the sub definition:

$sub = sub () { return $var };

This has been deprecated since Perl 5.22. [GH #17020]

Use of vec on strings with code points above 0xFF is forbidden

Such strings are represented internally in UTF-8, and vec is a bit-oriented operation that will likely give unexpected results on those strings. This was deprecated in perl 5.28.0.

Use of code points over 0xFF in string bitwise operators

Some uses of these were already illegal after a previous deprecation cycle. The remaining uses are now prohibited, having been deprecated in perl 5.28.0. See perldeprecation.

Sys::Hostname::hostname() does not accept arguments

This usage was deprecated in perl 5.28.0 and is now fatal.

Plain "0" string now treated as a number for range operator

Previously a range "0" .. "-1" would produce a range of numeric strings from "0" through "99"; this now produces an empty list, just as 0 .. -1 does. This also means that "0" .. "9" now produces a list of integers, where previously it would produce a list of strings.

This was due to a special case that treated strings starting with "0" as strings so ranges like "00" .. "03" produced "00", "01", "02", "03", but didn't specially handle the string "0". [GH #16770]

\K now disallowed in look-ahead and look-behind assertions

This was disallowed because it causes unexpected behaviour, and no-one could define what the desired behaviour should be. [GH #14638]

Performance Enhancements

Modules and Pragmata

Updated Modules and Pragmata

Removed Modules and Pragmata

Documentation

Changes to Existing Documentation

We have attempted to update the documentation to reflect the changes listed in this document. If you find any we have missed, open an issue at https://github.com/Perl/perl5/issues.

Additionally, the following selected changes have been made:

perldebguts

perlapi

perldiag

perlfunc

caller

Like __FILE__ and __LINE__, the filename and line number returned here may be altered by the mechanism described at "Plain Old Comments (Not!)" in perlsyn.

__FILE__

It can be altered by the mechanism described at "Plain Old Comments (Not!)" in perlsyn.

__LINE__

It can be altered by the mechanism described at "Plain Old Comments (Not!)" in perlsyn.

return

Now mentions that you cannot return from do BLOCK.

open

The open() section had been renovated significantly.

perlguts

perlhacktips

perlintro

perlipc

perlop

perlpod

perlport

perlreref

perlvar

perlapi, perlintern

perlxs

POSIX

Additionally, the following selected changes have been made:

Diagnostics

The following additions or changes have been made to diagnostic output, including warnings and fatal error messages. For the complete list of diagnostic messages, see perldiag.

New Diagnostics

New Errors

New Warnings

Changes to Existing Diagnostics

Utility Changes

perlbug

streamzip

Configuration and Compilation

Configure

Testing

Tests were added and changed to reflect the other additions and changes in this release. Furthermore, these significant changes were made:

Platform Support

Discontinued Platforms

Windows CE

Support for building perl on Windows CE has now been removed.

Platform-Specific Notes

Linux

cc will be used to populate plibpth if cc is clang. [GH #17043]

NetBSD 8.0

Fix compilation of Perl on NetBSD 8.0 with g++. [GH #17381]

Windows

  • The configuration for ccflags and optimize are now separate, as with POSIX platforms. [GH #17156]

  • Support for building perl with Visual C++ 6.0 has now been removed.

  • The locale tests could crash on Win32 due to a Windows bug, and separately due to the CRT throwing an exception if the locale name wasn't validly encoded in the current code page.

    For the second we now decode the locale name ourselves, and always decode it as UTF-8. [GH #16922]

  • t/op/magic.t could fail if environment variables starting with FOO already existed.

  • MYMALLOC (PERL_MALLOC) build has been fixed.

Solaris

  • Configure will now find recent versions of the Oracle Developer Studio compiler, which are found under /opt/developerstudio*.

  • Configure now uses the detected types for gethostby* functions, allowing Perl to once again compile on certain configurations of Solaris.

VMS

  • With the release of the patch kit C99 V2.0, VSI has provided support for a number of previously-missing C99 features. On systems with that patch kit installed, Perl's configuration process will now detect the presence of the header stdint.h and the following functions: fpclassify, isblank, isless, llrint, llrintl, llround, llroundl, nearbyint, round, scalbn, and scalbnl.

  • -Duse64bitint is now the default on VMS.

z/OS

Perl 5.32 has been tested on z/OS 2.4, with the following caveats:

  • Only static builds (the default) build reliably

  • When using locales, z/OS does not handle the LC_MESSAGES category properly, so when compiling perl, you should add the following to your Configure options

    ./Configure <other options> -Accflags=-DNO_LOCALE_MESSAGES

  • z/OS does not support locales with threads, so when compiling a threaded perl, you should add the following to your Configure options

    ./Configure <other Configure options> -Accflags=-DNO_LOCALE

  • Some CPAN modules that are shipped with perl fail at least one of their self-tests. These are: Archive::Tar, Config::Perl::V, CPAN::Meta, CPAN::Meta::YAML, Digest::MD5, Digest::SHA, Encode, ExtUtils::MakeMaker, ExtUtils::Manifest, HTTP::Tiny, IO::Compress, IPC::Cmd, JSON::PP, libnet, MIME::Base64, Module::Metadata, PerlIO::via-QuotedPrint, Pod::Checker, podlators, Pod::Simple, Socket, and Test::Harness.

    The causes of the failures range from the self-test itself is flawed, and the module actually works fine, up to the module doesn't work at all on EBCDIC platforms.

Internal Changes

Selected Bug Fixes

Obituary

Jeff Goff (JGOFF or DrForr), an integral part of the Perl and Raku communities and a dear friend to all of us, has passed away on March 13th, 2020. DrForr was a prominent member of the communities, attending and speaking at countless events, contributing to numerous projects, and assisting and helping in any way he could.

His passing leaves a hole in our hearts and in our communities and he will be sorely missed.

Acknowledgements

Perl 5.32.0 represents approximately 13 months of development since Perl 5.30.0 and contains approximately 220,000 lines of changes across 1,800 files from 89 authors.

Excluding auto-generated files, documentation and release tools, there were approximately 140,000 lines of changes to 880 .pm, .t, .c and .h files.

Perl continues to flourish into its fourth decade thanks to a vibrant community of users and developers. The following people are known to have contributed the improvements that became Perl 5.32.0:

Aaron Crane, Alberto Simões, Alexandr Savca, Andreas König, Andrew Fresh, Andy Dougherty, Ask Bjørn Hansen, Atsushi Sugawara, Bernhard M. Wiedemann, brian d foy, Bryan Stenson, Chad Granum, Chase Whitener, Chris 'BinGOs' Williams, Craig A. Berry, Dagfinn Ilmari Mannsåker, Dan Book, Daniel Dragan, Dan Kogai, Dave Cross, Dave Rolsky, David Cantrell, David Mitchell, Dominic Hargreaves, E. Choroba, Felipe Gasper, Florian Weimer, Graham Knop, Håkon Hægland, Hauke D, H.Merijn Brand, Hugo van der Sanden, Ichinose Shogo, James E Keenan, Jason McIntosh, Jerome Duval, Johan Vromans, John Lightsey, John Paul Adrian Glaubitz, Kang-min Liu, Karen Etheridge, Karl Williamson, Leon Timmermans, Manuel Mausz, Marc Green, Matthew Horsfall, Matt Turner, Max Maischein, Michael Haardt, Nicholas Clark, Nicolas R., Niko Tyni, Pali, Paul Evans, Paul Johnson, Paul Marquess, Peter Eisentraut, Peter John Acklam, Peter Oliver, Petr Písař, Renee Baecker, Ricardo Signes, Richard Leach, Russ Allbery, Samuel Smith, Santtu Ojanperä, Sawyer X, Sergey Aleynikov, Sergiy Borodych, Shirakata Kentaro, Shlomi Fish, Sisyphus, Slaven Rezic, Smylers, Stefan Seifert, Steve Hay, Steve Peters, Svyatoslav, Thibault Duponchelle, Todd Rinaldo, Tomasz Konojacki, Tom Hukins, Tony Cook, Unicode Consortium, VanL, Vickenty Fesunov, Vitali Peil, Yves Orton, Zefram.

The list above is almost certainly incomplete as it is automatically generated from version control history. In particular, it does not include the names of the (very much appreciated) contributors who reported issues to the Perl bug tracker.

Many of the changes included in this version originated in the CPAN modules included in Perl's core. We're grateful to the entire CPAN community for helping Perl to flourish.

For a more complete list of all of Perl's historical contributors, please see the AUTHORS file in the Perl source distribution.

Reporting Bugs

If you find what you think is a bug, you might check the perl bug database at https://github.com/Perl/perl5/issues. There may also be information at http://www.perl.org/, the Perl Home Page.

If you believe you have an unreported bug, please open an issue at https://github.com/Perl/perl5/issues. Be sure to trim your bug down to a tiny but sufficient test case.

If the bug you are reporting has security implications which make it inappropriate to send to a public issue tracker, then see "SECURITY VULNERABILITY CONTACT INFORMATION" in perlsec for details of how to report the issue.

Give Thanks

If you wish to thank the Perl 5 Porters for the work we had done in Perl 5, you can do so by running the perlthanks program:

perlthanks

This will send an email to the Perl 5 Porters list with your show of thanks.

SEE ALSO

The Changes file for an explanation of how to view exhaustive details on what changed.

The INSTALL file for how to build Perl.

The README file for general stuff.

The Artistic and Copying files for copyright information.