JPIH Weekly Status
From OMAPpedia
[edit] Jean's Weekly Status Report
jean.pihet@newoldbits.com j-pihet@ti.com
[edit] Links
- [1] http://marc.info/?l=linux-kernel&m=128195697205096&w=4 : patch proposal on LKML for additional PM trace events
- [2] http://omappedia.org/wiki/Power_Management_Debug_and_Profiling : PM Debug and Profiling wiki page
- [3] http://git.kernel.org/?p=linux/kernel/git/x86/linux-2.6-tip.git;a=commitdiff;h=74704ac6ea402d50c9543cb28247e6d9f521f7ae
- [4] http://marc.info/?l=linux-kernel&m=129201082403712&w=2 : re-spin of LKML patches (also cf. the 3 following patches).
[edit] Code merge estimates
- 1 Power management trace points API redifinition: ETA Oct 2010, cf. [4]. Merge ETA: 2.6.38 Jan 2011.
- 2 pytimechart tool support for clock, power domains: As soon as (1) is done => ETA Q1 2011.
- 3 pytimechart tool support for low power entry/exit latencies: As soon as (1) is done => ETA Q1 2011
[edit] Long term plan
- 1 Redefine the PM trace points API for cpufreq, cpuidle, suspend, clock & power domains: ETA Q4 2010 -Done-. Merge ETA: 2.6.38.
- 2 User space analysis tools (pytimechart) after (1): ETA Q1 2011
- 3 Rework the OMAP3 idle ASM code ETA 2.6.38 -Done-, code in DDRAM ETA Q4 2010, Merge ETA: Q1 2011. - Code in $I/$D (TBC): ETA Q1 2011.
- 4 Provide low power entry/exit latencies measurements: ETA Q4 2010 -Done-. Note: the system OFF (power supplies and external clocks OFF is not supported atm on l-o) is not included.
- 5 Investigate the OMAP4 HW tracing capability: ETA End of year 2010; Code for OMAP4 HW tracing: prototype code Q1 2011
- 6 Review the PM related patches: continuous task
- 7 Thermal management: Q1 2011 for OMAP3. OMAP4430/4440: TBC
[edit] Architecture discussions
- 1 Need to rework the OMAP3 idle ASM code: clean-up for readability; errata implementation, code in DDR + performance impacts measurements
- 2 OMAP4 HW tracing capability: Discussions with the HW architecture and SPI teams took place in Nice on Nov 30.
- HW capability: generation of traces from L3, clock and power modules; redirection of traces from STM to PTI (pins for trace receiver pod) or ETB (8KB memory buffer); messages filtering by source (master, channel), time (triggers), type; configurable sampling window per class (PM, CM1, CM2); CM can generate events vs time or statistical data + periodic reporting
- HW boards and trace: Pandaboard does not have any PTI interface; Blazeboard has it on board
- Compatible HW pods: LT32, XDS560v2
- Trace messages format and control: available in the EMU TRM. Traces format: STPv1 (OMAP4), STPv2 (partially on OMAP5, fully on OMAP6).
- 3 Thermal Management
- support for OMAP3 (from Nokia) is not in mainline and needs rework to adapt to the new OMAP devices framework
- support for OMAP4430: 1) -mandatory- DDR controller monitors the temp and generates an IRQ when the pre-defined threshold is reached, then an event is sent to the user space 2) -optional- the internal chip sensor needs to be monitored and an event reported to userspace
- support for OMAP4440: OMAP4430 + the HW monitors the temp sensor himself and generates an IRQ when the threshold is reached
- 4 EMIF statistical collectors
- On OMAP4 the EMIF can generate statistical data to the STM. A trace-compatible pod can be used to extract the data.
- ToDo: extract and parse the data w/o HW pod, using the ETB + parsing on the host machine. A link to the standard events framework is planned.
[edit] Week 2, 10 Jan 2011
- ASM code clean-up: code in DDR
- debugging on ES3 (RX51);
- measuring the gain in execution time and SRAM usage
- OMAP4 Blaze board received: starting up the board with Ubuntu distribution.
[edit] Week 1, 3 Jan 2011
- ASM code clean-up
- Clean-up code merged in
- Code in DDR: debugging on ES3 (RX51)
- PM trace API: code merged in, only the OMAP specific code remains. Patch proposed and being discussed on l-o ML
- Power domains latencies: code rework on-going, using the generic pm_qos framework
- OMAP4 Blaze board received: starting up the board with Ubuntu distribution.
[edit] Week 52, 27 Dec 2010
Low power mode week (i.e. Week Off ;-)
[edit] Week 51, 20 Dec 2010
- Rework of patches for the .38 merge window:
- ASM code clean-up: errata reviewed (submitted: Nishant), clean-up (submitted and reviewed OK, OK to go in l-o for .38), code in DDR (tests on-going, to be submitted after 2.6.38)
- Power domains latencies: code rework on-going, using the generic pm_qos framework
[edit] Week 50, 13 Dec 2010
- Rework of patches for the .38 merge window:
- ASM code clean-up: errata (submitted: Nishant), clean-up (submitted and reviews on-going), code in DDR (tests on-going, to be submitted after 2.6.38)
- Power domains latencies: measurements + code rework (on-going)
[edit] Week 49, 6 Dec 2010
- Rework of patches for the .38 merge window:
- PM trace API -Done-,
- ASM code clean-up: errata (submitted: Nishant), clean-up (submitted), code in DDR (tests on-going)
- Power domains latencies: measurements + code rework (on-going)
[edit] Week 48, 29 Nov 2010
- Discussions at TI Nice:
- Current status on PM for OMAP3/4
- Presentation and discussion in general OMAP4+ HW architecture
- Discussion about the tools support for OMAP4+: perf, ftrace. New possible tasks identified: EMIF stats
- OMAP4+ HW trace support: discussion with HW and SPI teams about the architecture and the implementation
- C-states latencies: discussion with team about the PM QoS
- ASM idle code clean-up:
- Nishant has pushed the reworked patches, rebasing the latest ASM clean-up on top
- discussion about the ASM code in DDR: only the DLL3 kick code for RET is needed in SRAM
- PM trace API rework: reviewing latest patches and helping the upstream work. Should go upstream soon via Ingo Molnar's tip tree. New version on-going
[edit] Week 47, 22 Nov 2010
- ASM idle code clean-up:
- reworking the ASM clean-up changes on top of Nishant's + testing => OK done locally. Nishant is pushing the reworked patches asap.
- reworking the patches from Vishwa
- PM trace API rework: reviewing latest patches and helping the upstream work. Should go upstream soon via Ingo Molnar's tip tree. New version on-going
- Power domains latencies measurements: updated the Wiki page with latest patches for GPT12 (for Beagle only). Measurements figures available. The use of those numbers in actual code needs to be discussed; proposed discussion next week at TI Nice.
- Investigation on latest l-o low power mode breakage for idle: Paul & Kevin proposed 2 patches which are fixing the problem.
- Preparation of next week's meeting at TI Nice.
[edit] Week 46, 15 Nov 2010
- ASM idle code clean-up:
- investigating the latest OMAP34xx erratas => done. Proposed the changes on l-o ML. Nishant replied with a bunch of erratas (from Nokia?) for 34xx and 36xx.
- reworking the ASM clean-up changes on top of Nishant's + testing
- reworking the patches from Vishwa
- PM trace API rework: reviewing latest patches and helping the upstream work. Should go upstream soon via Ingo Molnar's tip tree
- Investigation on latest l-o low power mode breakage for idle: the 'new worst latency' messages are interacting with the idle code, causing some modules in PER (UART2&3) hang. Investigation the root cause. Kevin has an experimental patch. The real solution is to convert the UARTs to HWMOD.
- reviewing the latest Cortex perf patches on l-a => Done
[edit] Week 45, 8 Nov 2010
- Implemented a way to trace the HW wake-up latencies, using GPT12. Since GPT12 is used a wake-up source it can be used as a stopwatch. A SW tweak is needed to let the timer count after wake-up.
- New patches and first messurements are on the wiki page http://omappedia.org/wiki/Power_Management_Device_Latencies_Measurement.
- Investigation on latest l-o low power mode breakage for idle: the 'new worst latency' messages are interacting with the idle code, causing some modules in PER (UART2&3) hang. Investigation is needed on the root cause.
[edit] Week 44, 1 Nov 2010
- Investigating a HW approach for tracing the OFF mode restore code: scope on the following signals:
- 32k/26MHz CLKOUT signals,
- SYS_OFF_MODE to PWIC,
- GPIOs for synching the HW trace with the SW trace points.
- First results gathered for OFF and RET mode. A wiki page at http://omappedia.org/wiki/Power_Management_Device_Latencies_Measurement has been created with the detailed results and implication in the code.
- Investigation on latest l-o low power mode breakage: a patch from Kevin's suspend-pm branch is needed to allow the low power modes.
- Investigation on the way to get detailed latency figures for all power domains.
- Day off on Tue
[edit] Week 43, 25 Oct 2010
- Located a Beagleboard B5, which needed a HW fix (package problem on U9 chip, known problem).
- Investigating a HW approach for tracing the OFF mode restore code: scope on the following signals:
- 32k/26MHz CLKOUT signals,
- SYS_OFF_MODE to PWIC,
- GPIOs for synching the HW trace with the SW trace points.
- First results gathered for OFF and RET mode. A wiki page will be created with the detailed results and implication in the code.
- Discussions about the power trace API still on-going, a compromise has been found: retaining old API while providing the new API for a while (couple of kernel versions). When the patches are merged in mainline the ARM/OMAP specific code can be submitted.
[edit] Week 42, 18 Oct 2010
- Now that the l-o supports OFF mode, the investigation on the trace points in the ASM idle code goes on.
- Investigated the OFF mode code path with the Lauterbach debugger, both on OMAP3EVM and on N900. Although the debugger can step through the restore code (running in DDR & SRAM), detailed timing information is not provided.
- Investigating a HW approach for tracing the OFF mode restore code: scope on 32k/26MHz CLKOUT signals and GPIOs from OFF code.
- Adding trace points in the idle code: adding trace points too early in the wake-up sequence (before the SYNC32K iclk is re-enabled in the PRCM restore function) causes an abort. New code OK.
- Discussions about the power trace API still on-going, a compromise has been found: retaining old API while providing the new API for a while (couple of kernel versions). When the patches are merged in mainline the ARM/OMAP specific code can be submitted.
[edit] Week 41, 11 Oct 2010
- Now that the l-o supports OFF mode, the investigation on the trace points in the ASM idle code goes on.
- Problem with trace points after the save_context function in ASM => problem was with the caches which are invalidated but not disabled.
- Adding trace points in the idle code: adding trace points too early in the wake-up sequence (before the SYNC32K iclk is re-enabled in the PRCM restore function) causes an abort. Discussion about the problem is on pm-domain-all, investigation on-going.
- New patches provided on pm-domain-all.
- Discussions about the power trace API still on-going.
[edit] Week 40, 4 Oct 2010
- Rework of the patches on LKML induced a discussion about the power trace points API. A new and more consistent API has been proposed but some user space tools are using the current API. A transitional patch is needed, proposal to come on LKML. Cf. [1]. New patches submitted and lots of reactions received => reworking the patches, cf. [4].
- Idle code in DDR: instead of having the ASM code to call the C functions, the opposite should be done. Proof of concept clean-up code on-going, to be discussed with all involved parties => a call is to be planned this week.
- Investigating why the latest l-o master does not support the OFF mode => Root cause found, fix sumbitted, reviewed and corrected by Kevin, tested on OMAP3EVM with OFF and RET.
- 1/2 day off on Friday
[edit] Week 39, 27 Sep 2010
- Patches on LKML induced a discussion about the power trace points API. A new and more consistent API has been proposed but some user space tools are using the current API. A transitional patch is needed, proposal to come on LKML. Cf. [1]. New patches submitted and lots of reactions received => reworking the patches, cf. [4].
- investigating ASM code profiling: Vishwa has patches to run the idle code from the DDR instead of internal SRAM. Most of the code is C code and the ASM routine (that calls WFI) calls the C functions directly. Adding trace points in this code should be trivial. The next steps are: 1) merge in DDR code, 2) add tracepoints in the new code.
- Idle code in DDR: instead of having the ASM code to call the C functions, the opposite should be done. Proof of concept clean-up code on-going, to be discussed with all involved parties => a call is to be planned this week.
- Problem with trace points atomicity (oops: scheduling while atomic) => investigation on-going.
- Adding trace points in the idle code: adding trace points too early in the wake-up sequence (before the SYNC32K iclk is re-enabled in the PRCM restore function) causes an abort. Discussion about the problem is on pm-domain-all, investigation on-going.
- 1/2 day off on Friday
- Concern about HW availability: no OMAP3/4 board at hand anymore, so there is an urgent need for OMAP3 then OMAP4 HW.
[edit] Week 38, 20 Sep 2010
- Part of the patches proposed on LKML has been merged in the tip branch of the kernel, cf. [3]
- Patches on LKML induced a discussion about the power trace points API. A new and more consistent API has been proposed but some user space tools are using the current API. A transitional patch is needed, proposal to come on LKML. Cf. [1]
- pytimechart: working on idle latency support. Prototype ready, cf. screenshots on wiki page [2]
- investigating ASM code profiling: Vishwa has patches to run the idle code from the DDR instead of internal SRAM. Most of the code is C code and the ASM routine (that calls WFI) calls the C functions directly. Adding trace points in this code should be trivial. The next steps are: 1) merge in DDR code, 2) add tracepoints in the new code.
- Idle code in DDR: instead of having the ASM code to call the C functions, the opposite should be done. Proof of concept clean-up code on-going, to be discussed with all involved parties => a call is foreseen in 2 weeks (Vishwa on leave until Oct 4th).
- Adding trace points in the idle code: adding trace points too early in the wake-up sequence (before the SYNC32K iclk is re-enabled in the PRCM restore function) causes an abort. Discussion about the problem is on pm-domain-all, investigation on-going.
[edit] Week 37, 13 Sep 2010
- Patches on LKML has been merged in the tip branch of the kernel, cf. [3]
- Patches on LKML induced a discussion about the power trace points API. A new and more consistent API will be proposed. The user space tools (only pytimechart for now) needs to change. Cf. [1]
- First patches to pytimechart merged in, cf. http://gitorious.com/pytimechart/. Next patches for clocks and power tracing are ready and will be submitted as soon as the kernel changes are merged in from LKML.
- pytimechart: working on idle latency support. Prototype ready, cf. screenshots on wiki page [2]
- investigating ASM code profiling: Vishwa has patches to run the idle code from the DDR instead of internal SRAM. Most of the code is C code and the ASM routine (that calls WFI) calls the C functions directly. Adding trace points in this code should be trivial. The next steps are: 1) merge in DDR code, 2) add tracepoints in the new code.
- Adding trace points in the idle code: adding trace points too early in the wake-up sequence (before the SYNC32K iclk is re-enabled in the PRCM restore function) causes an abort. Discussion about the problem is on-going on pm-domain-all
- Looking at OMAP4 code for the new tracepoints integration => OMAP4 PM code is in the kernel-omap4-next branch of the omapzoom.org git tree.
[edit] Week 36, 6 Sep 2010
- Re-submitting patches on LKML after discussion.
- pytimechart: working on idle latency support. Prototype ready, cf. screenshots on wiki page
- Tools: other tools would require modification: powertop, cpufrequtils? Others? Interaction on LKML on-going about it.
- investigating ASM code profiling: it is not easy since one has to access ftrace internal buffers to tamper the time stamps, also the ASM code needs some changes to record a time stamp which will be used in the trace events at wake-up (in C code) => After the discussion at the weekly call: most of the ASM code is being converted to C-code that runs from DDR, so adding tracepoints will be trivial.
- Low power mode code (ASM code in sleep34xx.S): investigating how to instrument the ASM code, which runs from internal SRAM only. IMO the code should be re-written as a clean-up and to provide execution from caches. Is this a good candidate for the next tasks? => (from the weekly call) clean-up is ok but running from caches on OMAP3 is risky and does not gain much; OMAP4 runs entirely from DDR. Conclusion: is it needed to add tracepoints in ASM for OMAP3? => Cf. point above.
- First patches to pytimechart merged in, cf. http://gitorious.com/pytimechart/. Next patches for clocks and power tracing are ready and will be submitted as soon as the kernel changes are merged in from LKML.
- Looking at OMAP4 code for the new tracepoints integration => OMAP4 PM code is in the kernel-omap4-next branch of the omapzoom.org git tree
[edit] Week 35, 30 Aug 2010
- Good feedback on LKML: patch should be accepted in principle. The new events definition are generic enough for other (all?) platforms to use them
- pytimechart tool update for some of the new events: local patches created, code is working OK. Some of the events still need to be supported -e.g. clock events- that are now handled as default events.
- pytimechart: working on idle latency support.
- Update of wiki page with latest code and pytimechart screenshots.
- Start investigation of HW tracing architecture, using the OMAP3/4 HW buffers
[edit] Week 34, 23 Aug 2010
- Interaction on l-o ML and LKML for patches submission. The idea is to split the changes from http://omappedia.org/wiki/Power_Management_Debug_and_Profiling#l-o_kernel_patch.2FRFC into generic changes (-> to LKML) and the OMAP2 specific changes (-> to l-o). No reply on LKML yet, still re-pinging for reaction
- Code test on board
- Update of wiki page
- Start investigation of HW tracing architecture, using the OMAP3/4 HW buffers
[edit] Week 33, 16 Aug 2010
- Code clean-up for patch submission. The new events definition (in include/trace/events/power.h) comes as a separate patch because it is generic code and will be posted on LKML. The rest of the code (in arch/arm) will be submitted on l-o when the generic code is accepted
- Generic code posted on LKML, cf. http://marc.info/?l=linux-kernel&m=128195697205096&w=4
- Generation of a recipe for the perf utility for Arago. Filesystem build and on-board testing.
- Updated the PM Debug & Profiling wiki page
[edit] Week 32, 9 Aug 2010
- Added the clocks and power domains events. The intention is to provide new generic events that all platforms can use
- Test of new events on board
- Evaluation of the pytimechart UI tool. The new events are displayed as generic events. Ideally the UI tool will need some changes for the new events specific (i.e. display clock names, power domain names and the status ...)
- Generation of an user space image, for PM testing. Arago is used. The build takes forever...
- Update of the Wiki page with details, code and UI screenshots
- Day off on Fri 13 Aug 2010
[edit] Week 31, 2 Aug 2010
- Investigate the latest PM code
- Read OMAP PM wiki pages
- Setup source tree and compiler
- Weekly call
- Investigate the latest perf/ftrace code, especially for Power Mgt. Some code exists for cpuidle and cpufreq for x86. The events are exported from the perf framework to ftrace, so both tools can be used to trace the vents.
- Prototype code for tracing cpufreq and cpuidle events on OMAP2/3/4
- Test prototype code on OMAP3EVM board
- Read new HWMOD code
- Write Wiki page for PM debug and profiling at http://omappedia.org/wiki/Power_Management_Debug_and_Profiling, added a link from the PM main page
