Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Work/dp catalog review updates #3191

Merged
merged 105 commits into from
Feb 10, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
26c8765
Starting on FDD for DpCatalog
timcanham Jan 19, 2024
041a450
More DpCatalog SDD edits
timcanham Jan 22, 2024
c4f6294
More refined DpCatalog requirements
timcanham Jan 23, 2024
91a363e
Starting prototype code
timcanham Jan 24, 2024
3e7d121
Working on DpCatalog build command
timcanham Jan 24, 2024
e1a5783
More DpCatalog work
timcanham Jan 26, 2024
fe96b1a
More DpCatalog work
timcanham Jan 29, 2024
666e77b
Merge from devel
timcanham Feb 20, 2024
f99d851
More work on reading DP headers
timcanham Feb 21, 2024
4394336
More work...
timcanham Feb 21, 2024
f3af664
Continued work
timcanham Feb 26, 2024
c17652f
Spelling fixes
timcanham Feb 27, 2024
06c3fa6
More spelling fixes
timcanham Feb 27, 2024
c80d844
First run of init unit test
timcanham Feb 27, 2024
2848241
One DP works
timcanham Feb 29, 2024
7b9be73
Refactor unit test helpers
timcanham Mar 1, 2024
e51a07e
Working on multiple DPs
timcanham Mar 1, 2024
da0700e
More work on multiple DPs
timcanham Mar 3, 2024
0a1176d
Fixed filesRead bug
timcanham Mar 5, 2024
7425848
Merge from fprime devel
timcanham Mar 5, 2024
1e3452f
More merge fixes
timcanham Mar 5, 2024
cdd6b71
Adding DPs to SignalGen
timcanham Mar 11, 2024
1f0d4ec
Merge from F Prime devel
timcanham Mar 29, 2024
a0748fe
Adding DP components
timcanham Mar 29, 2024
7753fdd
Added DP directory to DpWriter
timcanham Mar 29, 2024
977a6a6
First working version of Data Products!
timcanham Mar 30, 2024
d4a3a0e
Added missing FileDownlink connection
timcanham Apr 3, 2024
3797067
Added early proto warning to SDD.
timcanham Apr 8, 2024
dc5dd7b
Switched to Fw::FileNameString
timcanham Apr 10, 2024
f8ac1cc
Fixed unit test compile
timcanham Apr 10, 2024
018f4ad
Add missing override keyword
timcanham Apr 10, 2024
38a3fd6
Fix unit test override
timcanham Apr 10, 2024
745b19f
Fixed format specifiers
timcanham Apr 10, 2024
d8991d6
Reformat code
bocchino Apr 11, 2024
715cc47
Revise DpWriter uts
bocchino Apr 11, 2024
52168ce
Revise dp writer
bocchino Apr 11, 2024
e3bc890
Revise dp writer
bocchino Apr 11, 2024
e62571c
Removed extra ;
timcanham Apr 25, 2024
623653a
Merge remote-tracking branch 'upstream/devel' into work/DpCatalog
timcanham Apr 25, 2024
da1065b
Fix markdown broken links
timcanham Apr 25, 2024
3c17315
Updated SDD to force CI to run again
timcanham Apr 25, 2024
f2fc62a
Adding async DP request example
timcanham Apr 26, 2024
0f09e8e
Working on async example
timcanham Apr 26, 2024
94170ce
Still working on async DP
timcanham Apr 26, 2024
b7912f6
Remove test printf
timcanham May 1, 2024
fa579b9
Merge from F Prime devel
timcanham May 1, 2024
e59053c
Merge remote-tracking branch 'upstream/devel' into work/DpCatalog
timcanham May 1, 2024
c6de09b
Another merge with F Prime devel
timcanham May 1, 2024
a823572
Merge with mainline
timcanham Jun 11, 2024
45516af
Merge remote-tracking branch 'nasa/devel' into work/DpCatalog
timcanham Jun 28, 2024
9aeb644
Merge remote-tracking branch 'upstream/devel' into work/DpCatalog
timcanham Aug 8, 2024
d4f5aaf
Merge remote-tracking branch 'upstream/devel' into work/DpCatalog
timcanham Aug 22, 2024
7867da5
Working on binary tree implementation
timcanham Aug 24, 2024
5f8be61
Work in progress. Doesn't work yet
timcanham Aug 25, 2024
489b7c9
Merge remote-tracking branch 'upstream/devel' into work/DpCatalog
timcanham Aug 30, 2024
e9bc028
Still working
timcanham Sep 16, 2024
526eca7
Merge remote-tracking branch 'nasa/devel' into work/DpCatalog
timcanham Sep 16, 2024
1bddd64
debug launch config
timcanham Sep 16, 2024
fb0f13e
fixed pre-launch run step
timcanham Sep 16, 2024
a02dd45
Some progress on algo
timcanham Sep 19, 2024
595e440
2 and 3 nodes work
timcanham Sep 19, 2024
8ae835d
Refactored some code
timcanham Sep 19, 2024
4ee2c62
Tree traverse seems to work!
timcanham Sep 26, 2024
7ab7857
One transmitted test
timcanham Sep 26, 2024
108b908
Fixed NULL to nullptr
timcanham Sep 26, 2024
3fdb91a
Fixed spelling
timcanham Sep 26, 2024
96add80
Fixed memory alignment
timcanham Sep 26, 2024
d1219c9
Starting on state file
timcanham Oct 1, 2024
ed9a697
Fixed spelling - AGAIN
timcanham Oct 1, 2024
5f8a863
Added reading of the state file
timcanham Oct 2, 2024
ed94902
Added function to get state for DP from state file
timcanham Oct 2, 2024
3257d82
Added state file data pruning function
timcanham Oct 2, 2024
f0c3793
Code to update the state file
timcanham Oct 3, 2024
da133de
Debugging DpCat
timcanham Oct 3, 2024
a07e5d6
More work on DPs
timcanham Oct 8, 2024
29d6704
Working on state file
timcanham Oct 9, 2024
905b061
Added to spelling expect.txt
timcanham Oct 9, 2024
c6dc446
fix double 'the'
timcanham Oct 9, 2024
ea98bfc
Testing with DpManager/SignalGen
timcanham Oct 11, 2024
195cc44
merge from core devel; updates to DpCatalog
timcanham Oct 11, 2024
e4696b5
Merge remote-tracking branch 'nasa/devel' into work/DpCatalog
timcanham Oct 18, 2024
31da9b9
Merge remote-tracking branch 'nasa/devel' into work/DpCatalog
timcanham Nov 7, 2024
f101d8a
Some compile fixes
timcanham Nov 7, 2024
e1bc0e3
Adding more test cases
timcanham Nov 13, 2024
2fae6ec
Fixed downlink bug
timcanham Nov 13, 2024
3bd7436
Fix SignalGen unit test
timcanham Nov 14, 2024
d836bda
Updated SDD and some CI error fixes
timcanham Nov 14, 2024
90cf037
Fix SDD typo
timcanham Nov 14, 2024
84889de
Merge with F Prime devel
timcanham Nov 25, 2024
07bd01b
Working on docs
timcanham Nov 25, 2024
0e1329c
Docs/comments updates
timcanham Dec 11, 2024
3dc293e
Fix typo
timcanham Dec 11, 2024
7c49ae6
Merge from devel; fixed conflicts
timcanham Jan 21, 2025
b6ad892
Added random priority test
timcanham Jan 21, 2025
c4a474f
More random tests
timcanham Jan 21, 2025
1500dc5
Added stdc to spelling expect
timcanham Jan 21, 2025
0ef1c4d
Working on requested fixes
timcanham Jan 23, 2025
65a69ff
Merge from devel
timcanham Feb 8, 2025
da4eead
Added DP port tests to SignalGen unit test
timcanham Feb 8, 2025
889abcb
Updates to address review issues
timcanham Feb 9, 2025
d64584a
Fix assert argument
timcanham Feb 9, 2025
183d600
Fixed some more ut CI errors
timcanham Feb 9, 2025
ee248ff
Attempting to fix spelling
LeStarch Feb 10, 2025
eb081a0
Removed incorrect comment
timcanham Feb 10, 2025
e7ccc1f
Merge branch 'work/DpCatalog' of github.com:timcanham/fprime into wor…
timcanham Feb 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions Ref/SignalGen/test/ut/SignalGenTester.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ namespace Ref {
{
this->initComponents();
this->connectPorts();
this->m_reqDpBuff.set(this->m_dpBuff,sizeof(this->m_dpBuff));
}

SignalGenTester ::
Expand All @@ -46,5 +47,41 @@ namespace Ref {
invoke_to_schedIn(0, 0);
component.doDispatch();
ASSERT_TLM_Output_SIZE(1);
sendCmd_Dp(0,10,Ref::SignalGen_DpReqType::IMMEDIATE,1,1);
component.doDispatch();
// verify request for data product buffer
ASSERT_PRODUCT_GET_SIZE(1);
// run 2 cycles, should output data product on second
invoke_to_schedIn(0, 0);
ASSERT_PRODUCT_SEND_SIZE(1);

}

//! Handle a text event
void SignalGenTester::textLogIn(
FwEventIdType id, //!< The event ID
const Fw::Time& timeTag, //!< The time
const Fw::LogSeverity severity, //!< The severity
const Fw::TextLogString& text //!< The event string
) {
TextLogEntry e = { id, timeTag, severity, text };

printTextLogHistoryEntry(e, stdout);

}

Fw::Success::T SignalGenTester ::
productGet_handler(
FwDpIdType id,
FwSizeType dataSize,
Fw::Buffer& buffer
)
{
printf ("Component requested %" PRI_FwSizeType " bytes.\n",dataSize);
buffer.set(this->m_dpBuff,dataSize);
this->pushProductGetEntry(id, dataSize);
return Fw::Success::SUCCESS;
}


} // end namespace Ref
22 changes: 22 additions & 0 deletions Ref/SignalGen/test/ut/SignalGenTester.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,18 @@ namespace Ref {

private:

//! Handle a data product get from the component under test
//!
//! By default, (1) call pushProductGetEntry; (2) do not allocate a buffer
//! and return FAILURE. You can override this behavior, e.g., to call
//! pushProductGetEntry, allocate a buffer and return SUCCESS.
virtual Fw::Success::T productGet_handler(
FwDpIdType id, //!< The container ID (input)
FwSizeType dataSize, //!< The data size of the requested buffer (input)
Fw::Buffer& buffer //!< The buffer (output)
);


// ----------------------------------------------------------------------
// Variables
// ----------------------------------------------------------------------
Expand All @@ -75,6 +87,16 @@ namespace Ref {
//!
SignalGen component;

void textLogIn(
FwEventIdType id, //!< The event ID
const Fw::Time& timeTag, //!< The time
const Fw::LogSeverity severity, //!< The severity
const Fw::TextLogString& text //!< The event string
) override;

U8 m_dpBuff[1024];
Fw::Buffer m_reqDpBuff;

};

} // end namespace Ref
Expand Down
61 changes: 48 additions & 13 deletions Svc/DpCatalog/DpCatalog.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ namespace Svc {

// Do some assertion checks
FW_ASSERT(numDirs <= DP_MAX_DIRECTORIES, static_cast<FwAssertArgType>(numDirs));
FW_ASSERT(stateFile.length());
this->m_stateFile = stateFile;

// request memory for catalog which is DP_MAX_FILES * slot size.
Expand Down Expand Up @@ -130,6 +131,7 @@ namespace Svc {
void DpCatalog::resetBinaryTree() {
// initialize data structures in the free list
// Step 2 in memory partition (see configure() comments)
FW_ASSERT(this->m_memPtr);
this->m_freeListHead = static_cast<DpBtreeNode*>(this->m_memPtr);
for (FwSizeType slot = 0; slot < this->m_numDpSlots; slot++) {
// overlay new instance of the DpState entry on the memory
Expand All @@ -147,6 +149,7 @@ namespace Svc {
this->m_dpTree = nullptr;
// reset number of records
this->m_numDpRecords = 0;

LeStarch marked this conversation as resolved.
Show resolved Hide resolved
}

void DpCatalog::resetStateFileData() {
Expand Down Expand Up @@ -210,7 +213,9 @@ namespace Svc {
}

// reset the buffer for deserializing the entry
entryBuffer.setBuffLen(static_cast<Fw::Serializable::SizeType>(size));
Fw::SerializeStatus serStat = entryBuffer.setBuffLen(static_cast<Fw::Serializable::SizeType>(size));
// should always fit
FW_ASSERT(Fw::FW_SERIALIZE_OK == serStat,serStat);
entryBuffer.resetDeser();

// deserialization after this point should always work, since
Expand Down Expand Up @@ -290,8 +295,12 @@ namespace Svc {
// reset the buffer for serializing the entry
entryBuffer.resetSer();
// serialize the file directory index
entryBuffer.serialize(this->m_stateFileData[entry].entry.dir);
entryBuffer.serialize(this->m_stateFileData[entry].entry.record);
Fw::SerializeStatus serStat = entryBuffer.serialize(this->m_stateFileData[entry].entry.dir);
// Should always fit
FW_ASSERT(Fw::FW_SERIALIZE_OK == serStat,serStat);
serStat = entryBuffer.serialize(this->m_stateFileData[entry].entry.record);
// Should always fit
FW_ASSERT(Fw::FW_SERIALIZE_OK == serStat,serStat);
// write the entry
FwSignedSizeType size = entryBuffer.getBuffLength();
stat = stateFile.write(buffer, size);
Expand All @@ -308,9 +317,13 @@ namespace Svc {

void DpCatalog::appendFileState(const DpStateEntry& entry) {
FW_ASSERT(this->m_stateFileData);
FW_ASSERT(entry.dir < static_cast<FwIndexType>(this->m_numDirectories),
entry.dir,
static_cast<FwAssertArgType>(this->m_numDirectories)
);

// We will append state to the existing state file
// FIXME: Have to handle case where state file has partially transmitted
// TODO: Have to handle case where state file has partially transmitted
LeStarch marked this conversation as resolved.
Show resolved Hide resolved
// state already

// open the state file
Expand All @@ -323,13 +336,17 @@ namespace Svc {
}

// buffer for writing entries
BYTE buffer[sizeof(FwIndexType)+DpRecord::SERIALIZED_SIZE];
BYTE buffer[sizeof(entry.dir)+sizeof(entry.record)];
Fw::ExternalSerializeBuffer entryBuffer(buffer, sizeof(buffer));
// reset the buffer for serializing the entry
entryBuffer.resetSer();
// serialize the file directory index
entryBuffer.serialize(entry.dir);
entryBuffer.serialize(entry.record);
Fw::SerializeStatus serStat = entryBuffer.serialize(entry.dir);
// should fit
FW_ASSERT(serStat == Fw::FW_SERIALIZE_OK,serStat);
serStat = entryBuffer.serialize(entry.record);
// should fit
FW_ASSERT(serStat == Fw::FW_SERIALIZE_OK,serStat);
// write the entry
FwSignedSizeType size = entryBuffer.getBuffLength();
stat = stateFile.write(buffer, size);
Expand Down Expand Up @@ -375,6 +392,9 @@ namespace Svc {
// fill the binary tree with DP files
response = this->fillBinaryTree();
if (response != Fw::CmdResponse::OK) {
// clean up the binary tree
this->resetBinaryTree();
this->resetStateFileData();
return response;
}

Expand Down Expand Up @@ -514,8 +534,12 @@ namespace Svc {
this->getFileState(entry);

// insert entry into sorted list. if can't insert, quit
if (not this->insertEntry(entry)) {
bool insertedOk = this->insertEntry(entry);
Dismissed Show dismissed Hide dismissed
if (not insertedOk) {
this->log_WARNING_HI_DpInsertError(entry.record);
// clean up and return
this->resetBinaryTree();
this->resetStateFileData();
break;
}

Expand Down Expand Up @@ -567,7 +591,10 @@ namespace Svc {

// if the tree is empty, add the first entry
if (this->m_dpTree == nullptr) {
this->allocateNode(this->m_dpTree,entry);
bool goodInsert = this->allocateNode(this->m_dpTree,entry);
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
if (not goodInsert) {
return false;
}
// otherwise, search depth-first to sort the entry
} else {
// to avoid recursion, loop through a max of the number of available records
Expand Down Expand Up @@ -619,7 +646,8 @@ namespace Svc {
DpCatalog::CheckStat DpCatalog::checkLeftRight(bool condition, DpBtreeNode* &node, const DpStateEntry& newEntry) {
if (condition) {
if (node->left == nullptr) {
if (!this->allocateNode(node->left,newEntry)) {
bool allocated = this->allocateNode(node->left,newEntry);
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
if (not allocated) {
return CheckStat::CHECK_ERROR;
}
return CheckStat::CHECK_OK;
Expand All @@ -629,7 +657,8 @@ namespace Svc {
}
} else {
if (node->right == nullptr) {
if (!this->allocateNode(node->right,newEntry)) {
bool allocated = this->allocateNode(node->right,newEntry);
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
if (not allocated) {
return CheckStat::CHECK_ERROR;
}
return CheckStat::CHECK_OK;
Expand Down Expand Up @@ -701,7 +730,13 @@ namespace Svc {
static_cast<U32>(this->m_currentXmitNode->entry.record.getsize()),
this->m_currentXmitNode->entry.record.getpriority()
);
this->fileOut_out(0, this->m_currXmitFileName, this->m_currXmitFileName, 0, 0);
Svc::SendFileResponse resp = this->fileOut_out(0, this->m_currXmitFileName, this->m_currXmitFileName, 0, 0);
if (resp.getstatus() != Svc::SendFileStatus::STATUS_OK) {
// warn, but keep going since it may be an issue with this file but others could
// make it
this->log_WARNING_HI_DpFileSendError(this->m_currXmitFileName,resp.getstatus());
}

}

} // end sendNextEntry()
Expand Down Expand Up @@ -802,7 +837,7 @@ namespace Svc {

// check file status
if (resp.getstatus() != Svc::SendFileStatus::STATUS_OK) {
this->log_WARNING_HI_StateFileXmitError(this->m_currXmitFileName,resp.getstatus());
this->log_WARNING_HI_DpFileXmitError(this->m_currXmitFileName,resp.getstatus());
this->m_xmitInProgress = false;
this->cmdResponse_out(this->m_xmitOpCode,this->m_xmitCmdSeq,Fw::CmdResponse::EXECUTION_ERROR);
}
Expand Down
14 changes: 12 additions & 2 deletions Svc/DpCatalog/DpCatalog.fpp
Original file line number Diff line number Diff line change
Expand Up @@ -347,13 +347,23 @@ module Svc {
id 40 \
format "State file {} doesn't exist"

event StateFileXmitError(
event DpFileXmitError(
file: string size 80 @< The file
stat: Svc.SendFileStatus
) \
severity warning high \
id 41 \
format "Error transmitting DP file {}, stat {}. Halting xmit."
format "Error transmitting DP file {}, stat {}. Halting xmit." \
throttle 10

event DpFileSendError(
file: string size 80 @< The file
stat: Svc.SendFileStatus
) \
severity warning high \
id 42 \
format "Error sending DP file {}, stat {}. Halting xmit." \
throttle 10



Expand Down
Loading
Loading