The attached patch implements three optimizations:
1) Loops in flush_user_dcache_range_asm, flush_kernel_dcache_range_asm,
purge_kernel_dcache_range_asm, flush_user_icache_range_asm, and
flush_kernel_icache_range_asm are unrolled to reduce branch overhead.
2) The static branch prediction for cmpb instructions in pacache.S have
been reviewed and the operand order adjusted where necessary.
3) For flush routines in cache.c, we purge rather flush when we have no
context. The pdc instruction at level 0 is not required to write back
dirty lines to memory. This provides a performance improvement over the
fdc instruction if the feature is implemented.
Version 2 adds alternative patching.
The patch provides an average improvement of about 2%.
Signed-off-by: John David Anglin <dave.anglin@bell.net>
Signed-off-by: Helge Deller <deller@gmx.de>
The attached change removes the pte_inserted from pgtable.h. As a
result, we always flush the TLB entry when the associated page table
entry is changed.
This change doesn't impact performance signifcantly and it may catch
some cases where the TLB needs flushing but wasn't.
Signed-off-by: John David Anglin <dave.anglin@bell.net>
Signed-off-by: Helge Deller <deller@gmx.de>
- Convert pcitest build process to that used by other tools (iio, perf,
etc) (Gustavo Pimentel)
* remotes/lorenzo/pci/tools:
tools: PCI: Change pcitest compiling process
tools: PCI: Fix compilation warnings
- Detach VMD resources after stopping root bus to prevent orphan
resources (Jon Derrick)
* remotes/lorenzo/pci/vmd:
PCI: vmd: Detach resources after stopping root bus
- Fix Mediatek unchecked return value from devm_pci_remap_iospace()
(Gustavo A. R. Silva)
- Fix Mediatek endpoint/port matching logic (Honghui Zhang)
- Change Mediatek Root Port Class Code to PCI_CLASS_BRIDGE_PCI (Honghui
Zhang)
- Remove redundant Mediatek PM domain check (Honghui Zhang)
- Convert Mediatek to pci_host_probe() (Honghui Zhang)
- Fix Mediatek MSI enablement (Honghui Zhang)
- Add Mediatek system PM support for MT2712 and MT7622 (Honghui Zhang)
- Add Mediatek loadable module support (Honghui Zhang)
* remotes/lorenzo/pci/mediatek:
PCI: mediatek: Add loadable kernel module support
PCI: mediatek: Add system PM support for MT2712 and MT7622
PCI: mediatek: Fixup MSI enablement logic by enabling MSI before clocks
PCI: mediatek: Convert to use pci_host_probe()
PCI: mediatek: Remove the redundant dev->pm_domain check
PCI: mediatek: Fix class type for MT7622 to PCI_CLASS_BRIDGE_PCI
PCI: mediatek: Fix mtk_pcie_find_port() endpoint/port matching logic
PCI: mediatek: Fix unchecked return value
- Quirk Keystone K2G to limit MRRS to 256 (Kishon Vijay Abraham I)
- Update Keystone to use MRRS quirk for host bridge instead of open
coding (Kishon Vijay Abraham I)
- Refactor Keystone link establishment (Kishon Vijay Abraham I)
- Simplify and speed up Keystone link training (Kishon Vijay Abraham I)
- Remove unused Keystone host_init argument (Kishon Vijay Abraham I)
- Merge Keystone driver files into one (Kishon Vijay Abraham I)
- Remove redundant Keystone platform_set_drvdata() (Kishon Vijay Abraham
I)
- Rename Keystone functions for uniformity (Kishon Vijay Abraham I)
- Add Keystone device control module DT binding (Kishon Vijay Abraham I)
- Use SYSCON API to get Keystone control module device IDs (Kishon Vijay
Abraham I)
- Clean up Keystone PHY handling (Kishon Vijay Abraham I)
- Use runtime PM APIs to enable Keystone clock (Kishon Vijay Abraham I)
- Clean up Keystone config space access checks (Kishon Vijay Abraham I)
- Get Keystone outbound window count from DT (Kishon Vijay Abraham I)
- Clean up Keystone outbound window configuration (Kishon Vijay Abraham
I)
- Clean up Keystone DBI setup (Kishon Vijay Abraham I)
- Clean up Keystone ks_pcie_link_up() (Kishon Vijay Abraham I)
- Fix Keystone IRQ status checking (Kishon Vijay Abraham I)
- Add debug messages for all Keystone errors (Kishon Vijay Abraham I)
- Clean up Keystone includes and macros (Kishon Vijay Abraham I)
* remotes/lorenzo/pci/keystone:
PCI: keystone: Cleanup macros defined in pci-keystone.c
PCI: keystone: Reorder header file in alphabetical order
PCI: keystone: Add debug error message for all errors
PCI: keystone: Use ERR_IRQ_STATUS instead of ERR_IRQ_STATUS_RAW to get interrupt status
PCI: keystone: Cleanup ks_pcie_link_up()
PCI: keystone: Cleanup set_dbi_mode() and get_dbi_mode()
PCI: keystone: Cleanup outbound window configuration
PCI: keystone: Get number of outbound windows from DT
PCI: keystone: Cleanup configuration space access
PCI: keystone: Invoke runtime PM APIs to enable clock
PCI: keystone: Cleanup PHY handling
PCI: keystone: Use SYSCON APIs to get device ID from control module
dt-bindings: PCI: keystone: Add bindings to get device control module
PCI: keystone: Use uniform function naming convention
PCI: keystone: Remove redundant platform_set_drvdata() invocation
PCI: keystone: Merge pci-keystone-dw.c and pci-keystone.c
PCI: keystone: Remove unused argument from ks_dw_pcie_host_init()
PCI: keystone: Do not initiate link training multiple times
PCI: keystone: Move dw_pcie_setup_rc() out of ks_pcie_establish_link()
PCI: keystone: Use quirk to set MRRS for PCI host bridge
PCI: keystone: Use quirk to limit MRRS for K2G
- Fix Cadence PHY handling during probe (Alan Douglas)
- Signal Cadence Endpoint interrupts via AXI region 0 instead of last
region (Alan Douglas)
- Write Cadence Endpoint MSI interrupts with 32 bits of data (Alan
Douglas)
* remotes/lorenzo/pci/cadence:
PCI: cadence: Write MSI data with 32bits
PCI: cadence: Use AXI region 0 to signal interrupts from EP
PCI: cadence: Correct probe behaviour when failing to get PHY
- Cache VF config space size to optimize enumeration of many VFs
(KarimAllah Ahmed)
- Remove unnecessary <linux/pci-ats.h> include (Bjorn Helgaas)
* pci/virtualization:
PCI/IOV: Remove unnecessary include of <linux/pci-ats.h>
PCI/IOV: Use VF0 cached config space size for other VFs
- Add PCI support for peer-to-peer DMA (Logan Gunthorpe)
- Add sysfs group for PCI peer-to-peer memory statistics (Logan
Gunthorpe)
- Add PCI peer-to-peer DMA scatterlist mapping interface (Logan
Gunthorpe)
- Add PCI configfs/sysfs helpers for use by peer-to-peer users (Logan
Gunthorpe)
- Add PCI peer-to-peer DMA driver writer's documentation (Logan
Gunthorpe)
- Add block layer flag to indicate driver support for PCI peer-to-peer
DMA (Logan Gunthorpe)
- Map Infiniband scatterlists for peer-to-peer DMA if they contain P2P
memory (Logan Gunthorpe)
- Register nvme-pci CMB buffer as PCI peer-to-peer memory (Logan
Gunthorpe)
- Add nvme-pci support for PCI peer-to-peer memory in requests (Logan
Gunthorpe)
- Use PCI peer-to-peer memory in nvme (Stephen Bates, Steve Wise,
Christoph Hellwig, Logan Gunthorpe)
* pci/peer-to-peer:
nvmet: Optionally use PCI P2P memory
nvmet: Introduce helper functions to allocate and free request SGLs
nvme-pci: Add support for P2P memory in requests
nvme-pci: Use PCI p2pmem subsystem to manage the CMB
IB/core: Ensure we map P2P memory correctly in rdma_rw_ctx_[init|destroy]()
block: Add PCI P2P flag for request queue
PCI/P2PDMA: Add P2P DMA driver writer's documentation
docs-rst: Add a new directory for PCI documentation
PCI/P2PDMA: Introduce configfs/sysfs enable attribute helpers
PCI/P2PDMA: Add PCI p2pmem DMA mappings to adjust the bus offset
PCI/P2PDMA: Add sysfs group to display p2pmem stats
PCI/P2PDMA: Support peer-to-peer memory
- Differentiate between pciehp surprise and safe removal (Lukas Wunner)
- Remove unnecessary pciehp includes (Lukas Wunner)
- Drop pciehp hotplug_slot_ops wrappers (Lukas Wunner)
- Tolerate PCIe Slot Presence Detect being hardwired to zero to
workaround broken hardware, e.g., the Wilocity switch/wireless device
(Lukas Wunner)
- Unify pciehp controller & slot structs (Lukas Wunner)
- Constify hotplug_slot_ops (Lukas Wunner)
- Drop hotplug_slot_info (Lukas Wunner)
- Embed hotplug_slot struct into users instead of allocating it
separately (Lukas Wunner)
- Initialize PCIe port service drivers directly instead of relying on
initcall ordering (Keith Busch)
- Restore PCI config state after a slot reset (Keith Busch)
- Save/restore DPC config state along with other PCI config state (Keith
Busch)
- Reference count devices during AER handling to avoid race issue with
concurrent hot removal (Keith Busch)
- If an Upstream Port reports ERR_FATAL, don't try to read the Port's
config space because it is probably unreachable (Keith Busch)
- During error handling, use slot-specific reset instead of secondary
bus reset to avoid link up/down issues on hotplug ports (Keith Busch)
- Restore previous AER/DPC handling that does not remove and re-enumerate
devices on ERR_FATAL (Keith Busch)
- Notify all drivers that may be affected by error recovery resets (Keith
Busch)
- Always generate error recovery uevents, even if a driver doesn't have
error callbacks (Keith Busch)
- Make PCIe link active reporting detection generic (Keith Busch)
- Support D3cold in PCIe hierarchies during system sleep and runtime,
including hotplug and Thunderbolt ports (Mika Westerberg)
- Handle hpmemsize/hpiosize kernel parameters uniformly, whether slots
are empty or occupied (Jon Derrick)
- Remove duplicated include from pci/pcie/err.c and unused variable from
cpqphp (YueHaibing)
- Remove driver pci_cleanup_aer_uncorrect_error_status() calls (Oza
Pawandeep)
- Uninline PCI bus accessors for better ftracing (Keith Busch)
- Remove unused AER Root Port .error_resume method (Keith Busch)
- Use kfifo in AER instead of a local version (Keith Busch)
- Use threaded IRQ in AER bottom half (Keith Busch)
- Use managed resources in AER core (Keith Busch)
- Reuse pcie_port_find_device() for AER injection (Keith Busch)
- Abstract AER interrupt handling to disconnect error injection (Keith
Busch)
- Refactor AER injection callbacks to simplify future improvments (Keith
Busch)
* pci/hotplug:
PCI/AER: Refactor error injection fallbacks
PCI/AER: Abstract AER interrupt handling
PCI/AER: Reuse existing pcie_port_find_device() interface
PCI/AER: Use managed resource allocations
PCI/AER: Use threaded IRQ for bottom half
PCI/AER: Use kfifo_in_spinlocked() to insert locked elements
PCI/AER: Use kfifo for tracking events instead of reimplementing it
PCI/AER: Remove error source from AER struct aer_rpc
PCI/AER: Remove unused aer_error_resume()
PCI: Uninline PCI bus accessors for better ftracing
PCI/AER: Remove pci_cleanup_aer_uncorrect_error_status() calls
PCI: pnv_php: Use kmemdup()
PCI: cpqphp: Remove set but not used variable 'physical_slot'
PCI/ERR: Remove duplicated include from err.c
PCI: Equalize hotplug memory and io for occupied and empty slots
PCI / ACPI: Whitelist D3 for more PCIe hotplug ports
ACPI / property: Allow multiple property compatible _DSD entries
PCI/PME: Implement runtime PM callbacks
PCI: pciehp: Implement runtime PM callbacks
PCI/portdrv: Add runtime PM hooks for port service drivers
PCI/portdrv: Resume upon exit from system suspend if left runtime suspended
PCI: pciehp: Do not handle events if interrupts are masked
PCI: pciehp: Disable hotplug interrupt during suspend
PCI / ACPI: Enable wake automatically for power managed bridges
PCI: Do not skip power-managed bridges in pci_enable_wake()
PCI: Make link active reporting detection generic
PCI: Unify device inaccessible
PCI/ERR: Always report current recovery status for udev
PCI/ERR: Simplify broadcast callouts
PCI/ERR: Run error recovery callbacks for all affected devices
PCI/ERR: Handle fatal error recovery
PCI/ERR: Use slot reset if available
PCI/AER: Don't read upstream ports below fatal errors
PCI/AER: Take reference on error devices
PCI/DPC: Save and restore config state
PCI: portdrv: Restore PCI config state on slot reset
PCI: portdrv: Initialize service drivers directly
PCI: hotplug: Document TODOs
PCI: hotplug: Embed hotplug_slot
PCI: hotplug: Drop hotplug_slot_info
PCI: hotplug: Constify hotplug_slot_ops
PCI: pciehp: Reshuffle controller struct for clarity
PCI: pciehp: Rename controller struct members for clarity
PCI: pciehp: Unify controller and slot structs
PCI: pciehp: Tolerate Presence Detect hardwired to zero
PCI: pciehp: Drop hotplug_slot_ops wrappers
PCI: pciehp: Drop unnecessary includes
PCI: pciehp: Differentiate between surprise and safe removal
PCI: Simplify disconnected marking
- Fix ASPM link_state teardown on removal (Lukas Wunner)
- Fix misleading _OSC ASPM message (Sinan Kaya)
- Make _OSC optional for PCI (Sinan Kaya)
- Don't initialize ASPM link state when ACPI_FADT_NO_ASPM is set (Patrick
Talbert)
* pci/aspm:
PCI/ASPM: Do not initialize link state when aspm_disabled is set
PCI/ACPI: Allow _OSC presence to be optional for PCI
PCI/ACPI: Correct error message for ASPM disabling
PCI/ASPM: Fix link_state teardown on device removal
DMA needs to be cleaned up not only on timeout, but on all errors where
it has been setup before.
Fixes: 73e8b05283 ("i2c: rcar: add DMA support")
Signed-off-by: Wolfram Sang <wsa+renesas@sang-engineering.com>
Signed-off-by: Wolfram Sang <wsa@the-dreams.de>
Add an entry for the Broadcom STB I2C controller in the MAINTAINERS file.
Signed-off-by: Kamal Dasu <kdasu.kdev@gmail.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
[wsa: fixed sorting and a whitespace error]
Signed-off-by: Wolfram Sang <wsa@the-dreams.de>
Ingo writes:
"x86 fixes:
It's 4 misc fixes, 3 build warning fixes and 3 comment fixes.
In hindsight I'd have left out the 3 comment fixes to make the pull
request look less scary at such a late point in the cycle. :-/"
* 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/swiotlb: Enable swiotlb for > 4GiG RAM on 32-bit kernels
x86/fpu: Fix i486 + no387 boot crash by only saving FPU registers on context switch if there is an FPU
x86/fpu: Remove second definition of fpu in __fpu__restore_sig()
x86/entry/64: Further improve paranoid_entry comments
x86/entry/32: Clear the CS high bits
x86/boot: Add -Wno-pointer-sign to KBUILD_CFLAGS
x86/time: Correct the attribute on jiffies' definition
x86/entry: Add some paranoid entry/exit CR3 handling comments
x86/percpu: Fix this_cpu_read()
x86/tsc: Force inlining of cyc2ns bits
Ingo writes:
"scheduler fixes:
Two fixes: a CFS-throttling bug fix, and an interactivity fix."
* 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
sched/fair: Fix the min_vruntime update logic in dequeue_entity()
sched/fair: Fix throttle_list starvation with low CFS quota
Ingo writes:
"perf fixes:
Misc perf tooling fixes."
* 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
perf tools: Stop fallbacking to kallsyms for vdso symbols lookup
perf tools: Pass build flags to traceevent build
perf report: Don't crash on invalid inline debug information
perf cpu_map: Align cpu map synthesized events properly.
perf tools: Fix tracing_path_mount proper path
perf tools: Fix use of alternatives to find JDIR
perf evsel: Store ids for events with their own cpus perf_event__synthesize_event_update_cpus
perf vendor events intel: Fix wrong filter_band* values for uncore events
Revert "perf tools: Fix PMU term format max value calculation"
tools headers uapi: Sync kvm.h copy
tools arch uapi: Sync the x86 kvm.h copy
The powernv platform maintains 2 TCE tables for VFIO - a hardware TCE
table and a table with userspace addresses. These tables are radix trees,
we allocate indirect levels when they are written to. Since
the memory allocation is problematic in real mode, we have 2 accessors
to the entries:
- for virtual mode: it allocates the memory and it is always expected
to return non-NULL;
- fr real mode: it does not allocate and can return NULL.
Also, DMA windows can span to up to 55 bits of the address space and since
we never have this much RAM, such windows are sparse. However currently
the SPAPR TCE IOMMU driver walks through all TCEs to unpin DMA memory.
Since we maintain a userspace addresses table for VFIO which is a mirror
of the hardware table, we can use it to know which parts of the DMA
window have not been mapped and skip these so does this patch.
The bare metal systems do not have this problem as they use a bypass mode
of a PHB which maps RAM directly.
This helps a lot with sparse DMA windows, reducing the shutdown time from
about 3 minutes per 1 billion TCEs to a few seconds for 32GB sparse guest.
Just skipping the last level seems to be good enough.
As non-allocating accessor is used now in virtual mode as well, rename it
from IOMMU_TABLE_USERSPACE_ENTRY_RM (real mode) to _RO (read only).
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
We've been getting checksum errors involving small UDP packets, usually
59B packets with 1 extra non-zero padding byte. netdev_rx_csum_fault()
has been complaining that HW is providing bad checksums. Turns out the
problem is in pskb_trim_rcsum_slow(), introduced in commit 88078d98d1
("net: pskb_trim_rcsum() and CHECKSUM_COMPLETE are friends").
The source of the problem is that when the bytes we are trimming start
at an odd address, as in the case of the 1 padding byte above,
skb_checksum() returns a byte-swapped value. We cannot just combine this
with skb->csum using csum_sub(). We need to use csum_block_sub() here
that takes into account the parity of the start address and handles the
swapping.
Matches existing code in __skb_postpull_rcsum() and esp_remove_trailer().
Fixes: 88078d98d1 ("net: pskb_trim_rcsum() and CHECKSUM_COMPLETE are friends")
Signed-off-by: Dimitris Michailidis <dmichail@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
At least UDP / TCP stacks can now cook skbs with a tstamp using
MONOTONIC base (or arbitrary values with SCM_TXTIME)
Since loopback driver does not call (directly or indirectly)
skb_scrub_packet(), we need to clear skb->tstamp so that
net_timestamp_check() can eventually resample the time,
using ktime_get_real().
Fixes: 80b14dee2b ("net: Add a new socket option for a future transmit time.")
Fixes: fb420d5d91 ("tcp/fq: move back to CLOCK_MONOTONIC")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Willem de Bruijn <willemb@google.com>
Cc: Soheil Hassas Yeganeh <soheil@google.com>
Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
-----BEGIN PGP SIGNATURE-----
iQIcBAABAgAGBQJbypLRAAoJEAx081l5xIa+mMMP/1ywMk9LSQhECJrU58f9XAts
vOiFrtgy4HLxUw7Nj8PFV3tXTAhuNfITJ5az01ggLPpFH4KtVRtHUM9xMc7wRUJe
5seR6AntgiwjZ5VU036+BMD1j6c1cD7OA7BY2moio3PAbXv+wmQ2P3qwGl5pxzVR
20emwYgQSfkiThblI2MwNvV1os8Gy6bDO9vowVc1+vpYkWvCf5lEVvaZRsujZl1/
FvLamGVIs19VlNMUzlUFKqv8c54qnv1y7CpZtNNZEGDitT+/fPC1BmyyxTMaNl7Z
iMhfHwT4hFy9CS4EcLqFvGgKZSGrC9IzSjiCYWo/uXsqDMv8SrlH33adBlcwMM7q
5ECALrSLF0BKWuD3dU/GipYHB0FmEm9U6nbruHpO9k3HyayJ3+mU2NjvCncxzLFb
J/lywag+GeB3pcNUNIuvN4IIBl85ytJTUV+yU1DYlxb644cPighlLZqMcAJJg5kP
045ixmSKiPyLv2hVUn5DsrnzMx1S7j9HUZwzsg4BXuguYu7pz6p3ui3sjgZuN+vU
ryvpmM0YeySB6BfbVTqTNFzRtN6ozcy6KIu/TKNh1qudE46ktA0+/7NkZ+dN5HN4
RqZQarbCTq2x4mpOFWBFwdjl0ehhtcAFL9eKJoD55PxzbV/8P205gD5loUVQI+qi
Qwawgc3Mn3Z/PbZhrKtI
=ZlVF
-----END PGP SIGNATURE-----
Merge tag 'drm-fixes-2018-10-20-1' of git://anongit.freedesktop.org/drm/drm
Dave writes:
"drm fixes for 4.19 final (part 2)
Looked like two stragglers snuck in, one very urgent the pageflipping
was missing a reference that could result in a GPF on non-i915
drivers, the other is an overflow in the sun4i dotclock calcs
resulting in a mode not getting set."
* tag 'drm-fixes-2018-10-20-1' of git://anongit.freedesktop.org/drm/drm:
drm/sun4i: Fix an ulong overflow in the dotclock driver
drm: Get ref on CRTC commit object when waiting for flip_done
The first two patches fix handling of unsigned type, and handling
of a space before an ending semi-colon.
The third patch adds a selftest to test the processing of synthetic events.
-----BEGIN PGP SIGNATURE-----
iIoEABYIADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCW8pMwxQccm9zdGVkdEBn
b29kbWlzLm9yZwAKCRAp5XQQmuv6qn2jAQCV4leBtN9EAax4B9Mmy4e5oYGE0SDF
Qq0f/Zb1SLYbTwD/Wdo+mqOAc9EFYkrxRjvgufgqM4bOUufW4fOQnqqPnwI=
=GS2b
-----END PGP SIGNATURE-----
Merge tag 'trace-v4.19-rc8-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace
Steven writes:
"tracing: A few small fixes to synthetic events
Masami found some issues with the creation of synthetic events. The
first two patches fix handling of unsigned type, and handling of a
space before an ending semi-colon.
The third patch adds a selftest to test the processing of synthetic
events."
* tag 'trace-v4.19-rc8-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace:
selftests: ftrace: Add synthetic event syntax testcase
tracing: Fix synthetic event to allow semicolon at end
tracing: Fix synthetic event to accept unsigned modifier
Dmitry writes:
"Input updates for 4.19-rc8
Just an addition to elan touchpad driver ACPI table."
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input:
Input: elan_i2c - add ACPI ID for Lenovo IdeaPad 330-15IGM
commit b96672dd84 ("powerpc: Machine check interrupt is a non-
maskable interrupt") added a call to nmi_enter() at the beginning of
machine check restart exception handler. Due to that, in_interrupt()
always returns true regardless of the state before entering the
exception, and die() panics even when the system was not already in
interrupt.
This patch calls nmi_exit() before calling die() in order to restore
the interrupt state we had before calling nmi_enter()
Fixes: b96672dd84 ("powerpc: Machine check interrupt is a non-maskable interrupt")
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
The recent module relocation overflow crash demonstrated that we
have no range checking on REL32 relative relocations. This patch
implements a basic check, the same kernel that previously oopsed
and rebooted now continues with some of these errors when loading
the module:
module_64: x_tables: REL32 527703503449812 out of range!
Possibly other relocations (ADDR32, REL16, TOC16, etc.) should also have
overflow checks.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
This tests that a bctr (Branch to counter and link), ie. a function
call, to a wildly out-of-bounds address is handled correctly.
Some old kernel versions didn't handle it correctly, see eg:
"powerpc/slb: Force a full SLB flush when we insert for a bad EA"
https://lists.ozlabs.org/pipermail/linuxppc-dev/2017-April/157397.html
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
When we're running on Book3S with the Radix MMU enabled the page table
dump currently prints the wrong addresses because it uses the wrong
start address.
Fix it to use PAGE_OFFSET rather than KERN_VIRT_START.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
At boot we print the ranges we've mapped for the linear mapping and
what page size we've used. Also track whether the range is mapped
executable or not and display that as well.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
If we look closely at the logic in create_physical_mapping(), when
we're doing STRICT_KERNEL_RWX, we do the following steps:
- determine the gap from where we are to the end of the range
- choose an appropriate mapping_size based on the gap
- check if that mapping_size would overlap the __init_begin
boundary, and if not choose an appropriate mapping_size
We can simplify the logic by taking the __init_begin boundary into
account when we calculate the initial gap.
So add a next_boundary() function which tells us what the next
boundary is, either the __init_begin boundary or end. In future we can
add more boundaries.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
When we have CONFIG_STRICT_KERNEL_RWX enabled, we want to split the
linear mapping at the text/data boundary so we can map the kernel
text read only.
The current logic uses a goto inside the for loop, which works, but is
hard to reason about.
When we hit the goto retry case we set max_mapping_size to PMD_SIZE
and go back to the start.
Setting max_mapping_size means we skip the PUD case and go to the PMD
case.
We know we will pass the alignment and gap checks because the only
reason we are there is we hit the goto retry, and that is guarded by
mapping_size == PUD_SIZE, which means addr is PUD aligned and gap is
greater or equal to PUD_SIZE.
So the only part of the check that can fail is the mmu_psize_defs
check for the 2M page size.
If we just duplicate that check we can avoid the goto, and we get the
same result.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
When we have CONFIG_STRICT_KERNEL_RWX enabled, we want to split the
linear mapping at the text/data boundary so we can map the kernel
text read only.
Currently we always use a small page at the text/data boundary, even
when that's not necessary:
Mapped 0x0000000000000000-0x0000000000e00000 with 2.00 MiB pages
Mapped 0x0000000000e00000-0x0000000001000000 with 64.0 KiB pages
Mapped 0x0000000001000000-0x0000000040000000 with 2.00 MiB pages
This is because the check that the mapping crosses the __init_begin
boundary is too strict, it also returns true when we map exactly up to
the boundary.
So fix it to check that the mapping would actually map past
__init_begin, and with that we see:
Mapped 0x0000000000000000-0x0000000040000000 with 2.00 MiB pages
Mapped 0x0000000040000000-0x0000000100000000 with 1.00 GiB pages
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
When we have CONFIG_STRICT_KERNEL_RWX enabled, we want to split the
linear mapping at the text/data boundary so we can map the kernel text
read only.
But the current logic uses small pages for the entire text section,
regardless of whether a larger page size would fit. eg. with the
boundary at 16M we could use 2M pages, but instead we use 64K pages up
to the 16M boundary:
Mapped 0x0000000000000000-0x0000000001000000 with 64.0 KiB pages
Mapped 0x0000000001000000-0x0000000040000000 with 2.00 MiB pages
Mapped 0x0000000040000000-0x0000000100000000 with 1.00 GiB pages
This is because the test is checking if addr is < __init_begin
and addr + mapping_size is >= _stext. But that is true for all pages
between _stext and __init_begin.
Instead what we want to check is if we are crossing the text/data
boundary, which is at __init_begin. With that fixed we see:
Mapped 0x0000000000000000-0x0000000000e00000 with 2.00 MiB pages
Mapped 0x0000000000e00000-0x0000000001000000 with 64.0 KiB pages
Mapped 0x0000000001000000-0x0000000040000000 with 2.00 MiB pages
Mapped 0x0000000040000000-0x0000000100000000 with 1.00 GiB pages
ie. we're correctly using 2MB pages below __init_begin, but we still
drop down to 64K pages unnecessarily at the boundary.
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
When we have CONFIG_STRICT_KERNEL_RWX enabled, we try to split the
kernel linear (1:1) mapping so that the kernel text is in a separate
page to kernel data, so we can mark the former read-only.
We could achieve that just by always using 64K pages for the linear
mapping, but we try to be smarter. Instead we use huge pages when
possible, and only switch to smaller pages when necessary.
However we have an off-by-one bug in that logic, which causes us to
calculate the wrong boundary between text and data.
For example with the end of the kernel text at 16M we see:
radix-mmu: Mapped 0x0000000000000000-0x0000000001200000 with 64.0 KiB pages
radix-mmu: Mapped 0x0000000001200000-0x0000000040000000 with 2.00 MiB pages
radix-mmu: Mapped 0x0000000040000000-0x0000000100000000 with 1.00 GiB pages
ie. we mapped from 0 to 18M with 64K pages, even though the boundary
between text and data is at 16M.
With the fix we see we're correctly hitting the 16M boundary:
radix-mmu: Mapped 0x0000000000000000-0x0000000001000000 with 64.0 KiB pages
radix-mmu: Mapped 0x0000000001000000-0x0000000040000000 with 2.00 MiB pages
radix-mmu: Mapped 0x0000000040000000-0x0000000100000000 with 1.00 GiB pages
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Currently, we expect to be able to reach ftrace_caller() from all
ftrace-enabled functions through a single relative branch. With large
kernel configs, we see functions outside of 32MB of ftrace_caller()
causing ftrace_init() to bail.
In such configurations, gcc/ld emits two types of trampolines for mcount():
1. A long_branch, which has a single branch to mcount() for functions that
are one hop away from mcount():
c0000000019e8544 <00031b56.long_branch._mcount>:
c0000000019e8544: 4a 69 3f ac b c00000000007c4f0 <._mcount>
2. A plt_branch, for functions that are farther away from mcount():
c0000000051f33f8 <0008ba04.plt_branch._mcount>:
c0000000051f33f8: 3d 82 ff a4 addis r12,r2,-92
c0000000051f33fc: e9 8c 04 20 ld r12,1056(r12)
c0000000051f3400: 7d 89 03 a6 mtctr r12
c0000000051f3404: 4e 80 04 20 bctr
We can reuse those trampolines for ftrace if we can have those
trampolines go to ftrace_caller() instead. However, with ABIv2, we
cannot depend on r2 being valid. As such, we use only the long_branch
trampolines by patching those to instead branch to ftrace_caller or
ftrace_regs_caller.
In addition, we add additional trampolines around .text and .init.text
to catch locations that are covered by the plt branches. This allows
ftrace to work with most large kernel configurations.
For now, we always patch the trampolines to go to ftrace_regs_caller,
which is slightly inefficient. This can be optimized further at a later
point.
Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
If CONFIG_PPC_SPLPAR is not selected, steal_time will always
be NUL, so accounting it is pointless
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
scaled cputime is only meaningfull when the processor has
SPURR and/or PURR, which means only on PPC64.
Removing it on PPC32 significantly reduces the size of
vtime_account_system() and vtime_account_idle() on an 8xx:
Before:
00000000 l F .text 000000a8 vtime_delta
00000280 g F .text 0000010c vtime_account_system
0000038c g F .text 00000048 vtime_account_idle
After:
(vtime_delta gets inlined inside the two functions)
000001d8 g F .text 000000a0 vtime_account_system
00000278 g F .text 00000038 vtime_account_idle
In terms of performance, we also get approximatly 7% improvement on
task switch. The following small benchmark app is run with perf stat:
void *thread(void *arg)
{
int i;
for (i = 0; i < atoi((char*)arg); i++)
pthread_yield();
}
int main(int argc, char **argv)
{
pthread_t th1, th2;
pthread_create(&th1, NULL, thread, argv[1]);
pthread_create(&th2, NULL, thread, argv[1]);
pthread_join(th1, NULL);
pthread_join(th2, NULL);
return 0;
}
Before the patch:
Performance counter stats for 'chrt -f 98 ./sched 100000' (50 runs):
8228.476465 task-clock (msec) # 0.954 CPUs utilized ( +- 0.23% )
200004 context-switches # 0.024 M/sec ( +- 0.00% )
After the patch:
Performance counter stats for 'chrt -f 98 ./sched 100000' (50 runs):
7649.070444 task-clock (msec) # 0.955 CPUs utilized ( +- 0.27% )
200004 context-switches # 0.026 M/sec ( +- 0.00% )
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
scaled cputime is only meaningfull when the processor has
SPURR and/or PURR, which means only on PPC64.
In preparation of the following patch that will remove
CONFIG_ARCH_HAS_SCALED_CPUTIME on PPC32, this patch moves
all scaled cputing accounting logic into dedicated functions.
This patch doesn't change any functionality. It's only code
reorganisation.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
ipic_get_mcp_status() is used by targets implementing NMI
watchdog in target specific machine check handler in order
to known whether a machine check results from a watchdog
NMI reset.
In case of very early machine check, primary_ipic pointer
might not have been set yet, so ipic_get_mcp_status() needs
to check it for nullity before using it.
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
This patch fixes the following warnings (obtained with make W=1).
arch/powerpc/mm/slice.c: In function 'slice_range_to_mask':
arch/powerpc/mm/slice.c:73:12: error: comparison is always true due to limited range of data type [-Werror=type-limits]
if (start < SLICE_LOW_TOP) {
^
arch/powerpc/mm/slice.c:81:20: error: comparison is always false due to limited range of data type [-Werror=type-limits]
if ((start + len) > SLICE_LOW_TOP) {
^
arch/powerpc/mm/slice.c: In function 'slice_mask_for_free':
arch/powerpc/mm/slice.c:136:17: error: comparison is always true due to limited range of data type [-Werror=type-limits]
if (high_limit <= SLICE_LOW_TOP)
^
arch/powerpc/mm/slice.c: In function 'slice_check_range_fits':
arch/powerpc/mm/slice.c:185:12: error: comparison is always true due to limited range of data type [-Werror=type-limits]
if (start < SLICE_LOW_TOP) {
^
arch/powerpc/mm/slice.c:195:39: error: comparison is always false due to limited range of data type [-Werror=type-limits]
if (SLICE_NUM_HIGH && ((start + len) > SLICE_LOW_TOP)) {
^
arch/powerpc/mm/slice.c: In function 'slice_scan_available':
arch/powerpc/mm/slice.c:306:11: error: comparison is always true due to limited range of data type [-Werror=type-limits]
if (addr < SLICE_LOW_TOP) {
^
arch/powerpc/mm/slice.c: In function 'get_slice_psize':
arch/powerpc/mm/slice.c:709:11: error: comparison is always true due to limited range of data type [-Werror=type-limits]
if (addr < SLICE_LOW_TOP) {
^
Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
