Compatibility of heap_useNewlib_NXP.c witx newlib 4.3

[escherstair]

Based on what I see, the wrapper heap_useNewlib_NXP.c has been verified for newlib versions 2.5 - 4.2
https://github.com/DRNadler/FreeRTOS_helpers/blob/e325ec7025c68cb6affbdf4003c6c3383eeee126/heap_useNewlib_NXP.c#L70C43-L70C68

The latest releases ARM GCC 12.3.Rel1 and 13.2.Rel1 come with newlib 4.3.
@DRNadler do you know if the wrapper works with this release?

[DRNadler]

I'm using GCC 13.2.1 and newlib 4.3.0 in a project now - AOK.
Apologize I just haven't had the time to review/accept changes here...

[escherstair]

Thanks @DRNadler
Do you want me to submit a PR allowing newlib 4.3.0 for NXP?

[smanger1226]

@DRNadler We're using STM 1.18.0 and it comes with newlib v4.4.0. I applied your fix. However, I still see some issues. I am occasionally seeing Hard Faults in malloc_r and some stl containers when removing or inserting items. The fix drastically improved the system.

Do you have any suggestions?

[escherstair]

Hi @smanger1226
I have more or less the same feeling as yours, and I work on NXP + cJSON (not STL containers, but a lot of malloc_r)
The helper greatly improved the situation, but after a couple of days of stress test, I get a fault.
I've been trying to narrow-down what happens, but it's not easy because I work on a HMP (iMX8M-Plus) and I cannot debug the Cortex-M alone.

[DRNadler]

I've used these tools forever with newlib/GCC/C++, as have hundreds of others, for a large number of products - without faults. Which of course certainly doesn't mean there are no issues...

Some generic suggestions:

• first, sounds silly but very often the issue: go back and triple-check stack use of all tasks and ISR stack
• ST has a long history of incorrect builds of newlib, use the ARM toolchain and not the one from ST (which has sometimes had incorrect build of reentrancy structures and support).
• C++ exceptions are not handled properly for thread safety - longtime issue outstanding with ARM not fixed last I checked:
  https://bugs.launchpad.net/gcc-arm-embedded/+bug/1905459
• If possible stress-test your application code in a host environment (ie build for PC with stubs). More complete development environment usually has better memory diagnostics (I've found many, many issues this way, for example some covered in this course: https://www.youtube.com/watch?v=pvoBpfK7iEQ).

Sorry I can't offer more help without details of your applications and environments!

[escherstair]

• first, sounds silly but very often the issue: go back and triple-check stack use of all tasks and ISR stack

I check stack use of all tasks calling uxTaskGetStackHighWaterMark(), but if I'm right I cannot check ISR stack because this requires that my port implements Stack Overflow Detection - Method 3.
Is this true?
As far as I understand, the FreeRTOS port for iMX8M-Plus doesn't support this.

[smanger1226]

I have checked the stacks, and my tasks usage is under 80%. I don't think there is an overflow problem. Since, I'm using newlib 4.4.0 with STM v1.18.0, I'm just looking for some advice. I'm going to continue doing some stress testing. Here's an overview of my testing, I have an osMemoryPool that contains 10 FatFs FIL *s. I have the memory pool wrapped in a class that has an std::set that contains any active FIL *. The use for this pool is so that we can flush/close all of the open files in case of power loss. I have 2 threads that are opening, writing, 'closing` a file many times. I see a HardFault on the commented out code, usually within a minute or two.

`FIL *FileIOFILMemoryPool::Alloc(const uint32_t timeout)
{
FIL *fp = TypedMemoryPool::Alloc(timeout);

// if (fp != nullptr) {
// _pMutex.TryAcquireMutex();
// _allocList.insert(reinterpret_cast<uint32_t>(fp));
// _pMutex.TryReleaseMutex();
// }

return (fp);

}

osStatus_t FileIOFILMemoryPool::Free(FIL *fp)
{
// _pMutex.TryAcquireMutex();
// _allocList.erase(reinterpret_cast<uint32_t>(fp));
// _pMutex.TryReleaseMutex();
//
return (TypedMemoryPool::Free(fp));
}

`

the thread function looks like this:
`void FIOThreadedWriter::_threadFunc(void *context)
{
FIOThreadedWriter *pObj = reinterpret_cast<FIOThreadedWriter *>(context);

if (pObj == nullptr) {
	pObj->_lc.Critical("thread: %s is nullptr",  osThreadGetName(pObj->_tId));
	osThreadExit();
	return;
}

FileIO f(pObj->_path);
IoReturnStatus status = IoReturnStatus::kUnknown;

for (uint32_t i = 0; i < pObj->_numWrites; i++) {
	size_t bw = 0;

	status = f.Open(FileMode::kAppendForWriting);
	status = f.Write(sizeof(i), reinterpret_cast<const uint8_t *>(&i), bw);
	f.Close();

	pObj->_lc.Info("numWrites: %u", i);
	osDelay(1);
}

pObj->_lc.Info("thread: %u is exiting",  pObj->_tId);
pObj->_complete = true;
osThreadExit();
return;

}
`

If you can see any problems, let me know.

[smanger1226]

Also as a follow up, I'm seeing these two warnings when building the code:

Description Resource Path Location Type
'prvHeapInit' defined but not used [-Wunused-function] heap_4.c /ControlBrd1_CM7/Middlewares/Third_Party/FreeRTOS line 333 C/C++ Problem
'prvInsertBlockIntoFreeList' defined but not used [-Wunused-function] heap_4.c /ControlBrd1_CM7/Middlewares/Third_Party/FreeRTOS line 381 C/C++ Problem

[DRNadler]

@smanger1226 - I'm confused - You are using both heap_useNewlib and heap4? That doesn't sound right???

[escherstair]

@smanger1226 if you use heap_useNewlib_NXP.c you must exclude from your build every heap_X.c that is shipped with FreeRTOS.
I double checked this in my project with the build logs and map files.
I had to manually chnage the cmake recipe to get this result.
I don't know what can happen if heap_useNewlib_NXP.c and heap_X.c are both built into your application.

[smanger1226]

I commented out the functions in heap_4.c that had multiple definitions. I just ran my test with the entire file commented out and it makes no difference.

[DRNadler]

@smanger1226 A few things puzzle me about your code regarding lock semantics for access to file pointers:

• One mutex protects all file operations?
• I do not understand your commented-out mutex acquisition/release. In my C++ code I always use a RAII pattern (ctor does not return until it acquires mutex, dtor releases mutex ) as it is exception-safe and rather more clear.
• Where is the mutex acquisition prior writing?
• Why are leading underscores used? Confusing though has nothing to do with the problem...
  Don't know if any of these are related to your problem...

[smanger1226]

I wanted to give you an update. There was a problem in our wrapper class for the CMSIS Mutex which was allowed multiple threads to operate on the stl container <std::set>. That is what was causing the hard faults. After I fixed that problem, I have seen no other issue with hard faults.

I'm satisfied that your patch works with this setup: STMCubeIDE 1.8.0 and newlib 4.4.0.

I saw malloc and free errors within 30 seconds during my multi-threaded stress testing. With your patch, I have run the stress testing for several days without any issues at all.

Thanks!

[DRNadler]

@smanger1226 - Thanks for letting me know.
Here's an example of a trouble-free way to grab and release mutex:

/// RAI (binary or mutex) semaphore protection for a code block (ctor takes, dtor gives)
class C_GrabAndReleaseSemaphore {
    SemaphoreHandle_t semaphoreHandle;
  public:
    inline C_GrabAndReleaseSemaphore(SemaphoreHandle_t semaphoreHandle_):
        semaphoreHandle(semaphoreHandle_)
        { xSemaphoreTake(semaphoreHandle, portMAX_DELAY); };
    inline ~C_GrabAndReleaseSemaphore()
        { xSemaphoreGive(semaphoreHandle); };
  private:
    C_GrabAndReleaseSemaphore(); // not implemented
    C_GrabAndReleaseSemaphore(const C_GrabAndReleaseSemaphore&); // not implemented
    C_GrabAndReleaseSemaphore & operator=(const C_GrabAndReleaseSemaphore&); // not implemented
};
typedef C_GrabAndReleaseSemaphore C_GrabAndReleaseMutex; // synonym; FreeRTOS uses same handle and API for semaphore and mutex...

Then you code any block messing with your protected resource like:

{
  C_GrabAndReleaseMutex t(global_mutex_handle);
  safe operation;
}

Explicit take and give is really error-prone...

Glad you got it working!
Best Regards, Dave