Hello, i tried to run BTS (<a href="https://www.bac-test.com/bacnet-test-suite-bts/" r

Bacnet testing with BTS about bacnet-stack HOT 8 OPEN

ErikasMasaitis commented on July 22, 2024

Bacnet testing with BTS

from bacnet-stack.

Comments (8)

BACnetEd commented on July 22, 2024

Gents I am concerned about anyone wasting time here - mainly due to "false positives". Our BACnet Test Suite has been used to test many (instances of B-ASC/B-BC) BACnet implementations, including many based on this stack, for successful BTL certifications. However, although it has been used for many non-BTL implementations, it has never been used to test a *router* through BTL... at least not completely... not yet. However, it is being used at the moment for a project that is about 95% of the way through BTL router testing, so it is nearly there. So, I will check Erikas' list against a 'good' router list and make sure that the BTS is consistent and provide feedback. And if anyone has a particular question, or wishes to challenge, any of the tests, please let me know. Steve, if you wish us to take this offline, let me know and we can do so.

…

On Wed, Oct 18, 2023 at 7:11 AM Erikas Masaitis ***@***.***> wrote: Hello, i tried to run BTS (https://www.bac-test.com/bacnet-test-suite-bts/), got a lot of failed tests. Wanted to know if there is any plans to test Bacnet stack with BTS or similar test benches? I am using example bacnet router application: Fail BC 135.1: 10.2.2.2.1 - Respond to Who-Is-Router-To-Network with I-Am-Router-To-Network when no Network Number specified Fail BC 135.1: 10.2.2.2.2-A - Respond to Who-Is-Router-To-Network with I-Am-Router-To-Network with known Network Number Fail BC 135.1: 10.2.2.2.2-B - Respond to Who-Is-Router-To-Network with I-Am-Router-To-Network with known Network Number - Requires BTT Transponder Fail BC 135.1: 10.2.2.2.3-A - Nearside Network Number is Specified and the Router Does Not Respond Fail BC 135.1: 10.2.2.7.1-B - Reject - Unknown Network Fail BC 135.1: 10.2.2.7.2 - Unknown Network Layer Message Type Fail BC 135.1: 10.2.3.6.1-B - Failed Attempt To Locate Downstream Routers Fail BC 135.1: 10.2.4.8 - Remote Broadcast to local Network_Number should be ignored Fail BC 135.1: 10.2.X2 - Routers Execute What-Is-Network-Number Fail BC 135.1: 10.8.2.1.2 - Respond to Who-Is-Router-To-Network with I-Am-Router-To-Network with known Network Number Fail BC 135.1: 10.8.2.2.2 - Unknown Network Layer Message Type Fail BC 135.1: 10.8.3.5.1 - Unknown Network - Ensure Reject not issued by Router if Broadcast Fail BITS : BIT00300 - Issuing Reject-Message-To-Network Unrecognised Service Fail BITS : BIT00305-B - Attempt to overwrite Accessible Route in Routing Table with I-Am-Router Fail BITS : BIT00309 - Rediscover all Devices via Remote Broadcast Who-Is Fail BITS : BIT00310 - Router Table vs Accessible Networks test Fail BITS : BIT01300 - Confirm Broadcast Distribution Table has Own IP Fail BITS : BIT01802 - BVLL should ignore some Broadcast messages Pass BC 135.1: 10.2.2.7.3-C - Unknown Network Layer Message Type Directed to FarSide for someone else - No BTT Pass BC 135.1: 10.2.2.7.3-D - Unknown Network Layer Message Type Local Broadcast - No BTT Pass BC 135.1: 10.2.2.7.3-E - Unknown Network Layer Message Type Global Broadcast - No BTT Pass BC 135.1: 10.2.2.7.3-F - Unknown Network Layer Message Type Remote Broadcast - No BTT Pass BC 135.1: 14.3.X1 - v17 Write-BDT service is required to return Write-BDT-NAK Pass BC 135.1: 14.6.1 - Execute Read Foreign Device Table Pass BC 135.1: 14.6.5 - Delete Foreign Device Table Entry Which Should Be Rejected Pass BC 135.1: 14.6.6 - Delete Foreign Device Table Entry Pass BITS : BIT00018 - Check Port ID Pass BITS : BIT00302-A - Check for Duplicate Router Table Entries Pass BITS : BIT00302-B - Check for Duplicate I-Am-Router Entries Pass BITS : BIT00303 - Check that there is only one (directly connected) Routing Entry per port Pass BITS : BIT00305-A - Attempt to add Duplicate Routing Entry using Init-Routing-Table Pass BITS : BIT00306 - Directed I-Am with Broadcast DADR should not be reflected Pass BITS : BIT00308 - Rediscover all Routers via Remote Broadcast Who-Is-Router and a lot of ignored tests..... — Reply to this email directly, view it on GitHub <#515>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEBER5AD5UWJVHBHVQP4OTX75XH3AVCNFSM6AAAAAA6E6B7N6VHI2DSMVQWIX3LMV43ASLTON2WKOZRHE2DQOJRG4YTAMI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- *Edward Hague**BACnet Interoperability Testing Services, Inc* +1 (650) 305 1395 155 Fleet Street, Portsmouth, NH 03801 www.bac-test.com LinkedIn: http://www.linkedin.com/in/edwardhague

from bacnet-stack.

ErikasMasaitis commented on July 22, 2024

Hello again, have you looked at tests?

from bacnet-stack.

BACnetEd commented on July 22, 2024

Hello Erikas - sorry, I thought I did answer..

I did run all the tests against some commercial routers and the tests all appear to be valid. In other words, no false alarms are suspected.

Please proceed accordingly. If you do come across an issue that is not clear / seems incorrect, please don't hesitate to contact me on this forum, for the benefit of all, else at [email protected].

from bacnet-stack.

josephtoutoo commented on July 22, 2024

Is anyone did BTL Certification using this stack. if you have information. Could you please provide information.

from bacnet-stack.

BACnetEd commented on July 22, 2024

Hi Joseph All I can say here is we have helped multiple customers who have used this stack get their products through BTL successfully. BUT - this stack has missing properties here and there, and these need to be added before submittal. This is straightforward, but it can take a surprisingly long time to get up the BACnet learning curve (if you are not there already) and implement them all. The stack itself is a bit of a moving target, because new objects, properties and services are continually being added, and the new ones, because they are new, may have even more missing features. You really have to draw a line under what Protocol Revision, profile and BIBBs you intend your product to have, figure out what is missing, implement those, and go from there. You cannot go too far back in Protocol Revisions (BTL has a limit: https://bacnetwiki.com/index.php?title=Minimum_Protocol_Revision), but by no means do you have to implement the very latest. In terms of 'code quality' of what is in this implementation - no complaints. Things could be optimised here and there (e.g. event driven COVs rather than an 'idle loop'), but for what it is - primarily a simple-as-possible implementation/starting point - it is just fine. And brace yourself - you are likely in for a $10K++ bill from BTL, even for a relatively simple device. Ed ***@***.*** www.bac-test.com

…

On Sat, Jan 20, 2024 at 2:37 AM josephtoutoo ***@***.***> wrote: Is anyone did BTL Certification using this stack. if you have information. Could you please provide information. — Reply to this email directly, view it on GitHub <#515 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEBER7NZMQB4WFBKXRRR2TYPMUXFAVCNFSM6AAAAAA6E6B7N6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBRGY2DGNJVGU> . You are receiving this because you commented.Message ID: ***@***.***>

-- *Edward Hague**BACnet Interoperability Testing Services, Inc* +1 (650) 305 1395 155 Fleet Street, Portsmouth, NH 03801 www.bac-test.com LinkedIn: http://www.linkedin.com/in/edwardhague

from bacnet-stack.

josephtoutoo commented on July 22, 2024

Hello Edward, Could you please mention the customer if you have any info. Could you please mention the missing properties . We are looking for protocol revision 19. We are working on new product. Do you have any code quality reports - static analysis report. Cyclo metric complexity. And other metrics. Thank you, Joseph toutoo.

…

On Sun, 21 Jan, 2024, 8:49 pm BACnetEd, ***@***.***> wrote: Hi Joseph All I can say here is we have helped multiple customers who have used this stack get their products through BTL successfully. BUT - this stack has missing properties here and there, and these need to be added before submittal. This is straightforward, but it can take a surprisingly long time to get up the BACnet learning curve (if you are not there already) and implement them all. The stack itself is a bit of a moving target, because new objects, properties and services are continually being added, and the new ones, because they are new, may have even more missing features. You really have to draw a line under what Protocol Revision, profile and BIBBs you intend your product to have, figure out what is missing, implement those, and go from there. You cannot go too far back in Protocol Revisions (BTL has a limit: https://bacnetwiki.com/index.php?title=Minimum_Protocol_Revision), but by no means do you have to implement the very latest. In terms of 'code quality' of what is in this implementation - no complaints. Things could be optimised here and there (e.g. event driven COVs rather than an 'idle loop'), but for what it is - primarily a simple-as-possible implementation/starting point - it is just fine. And brace yourself - you are likely in for a $10K++ bill from BTL, even for a relatively simple device. Ed ***@***.*** www.bac-test.com On Sat, Jan 20, 2024 at 2:37 AM josephtoutoo ***@***.***> wrote: > Is anyone did BTL Certification using this stack. if you have information. > Could you please provide information. > > — > Reply to this email directly, view it on GitHub > < #515 (comment)>, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAEBER7NZMQB4WFBKXRRR2TYPMUXFAVCNFSM6AAAAAA6E6B7N6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBRGY2DGNJVGU> > . > You are receiving this because you commented.Message ID: > ***@***.***> > -- *Edward Hague**BACnet Interoperability Testing Services, Inc* +1 (650) 305 1395 155 Fleet Street, Portsmouth, NH 03801 www.bac-test.com LinkedIn: http://www.linkedin.com/in/edwardhague — Reply to this email directly, view it on GitHub <#515 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEVJ36MB5YCNNSNMZSOC3HLYPUWW3AVCNFSM6AAAAAA6E6B7N6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBSGY3DQNJSGA> . You are receiving this because you commented.Message ID: ***@***.***>

from bacnet-stack.

BACnetEd commented on July 22, 2024

Joseph Answers inline Ed On Sun, Jan 21, 2024 at 3:34 PM josephtoutoo ***@***.***> wrote:

Hello Edward, Could you please mention the customer if you have any info.

EKH: Sorry, we have NDAs in place and can't mention "who"

Could you please mention the missing properties .

EKH: I don't have a complete list on-hand. And the properties are always a combination of what your selection of profile/BIBBS/Protocol Revision. Our BACnet Test Suite will do this for you. Let me know if you are interested in purchasing, but offline - let's keep this conversation in the forum on a non-commercial basis from now on. Contact me: ***@***.***

We are looking for protocol revision 19.

EKH: This is good for now. It will be acceptable by BTL until Jan 2027 per https://bacnetwiki.com/index.php?title=Minimum_Protocol_Revision

We are working on new product. Do you have any code quality reports - static analysis report. Cyclo metric complexity. And other metrics.

EKH: I don't, but someone else with the project loaded may have. Let's wait. Bear in mind that a lot of the folks involved in this stack are going to be at AHR for this coming week.

…

Thank you, Joseph toutoo. On Sun, 21 Jan, 2024, 8:49 pm BACnetEd, ***@***.***> wrote: > Hi Joseph > > All I can say here is we have helped multiple customers who have used > this stack get their products through BTL successfully. > > BUT - this stack has missing properties here and there, and these need to > be added before submittal. This is straightforward, but it can take a > surprisingly long time to get up the BACnet learning curve (if you are not > there already) and implement them all. > > The stack itself is a bit of a moving target, because new objects, > properties and services are continually being added, and the new ones, > because they are new, may have even more missing features. You really have > to draw a line under what Protocol Revision, profile and BIBBs you intend > your product to have, figure out what is missing, implement those, and > go from there. You cannot go too far back in Protocol Revisions (BTL has a > limit: https://bacnetwiki.com/index.php?title=Minimum_Protocol_Revision), > but by no means do you have to implement the very latest. > > In terms of 'code quality' of what is in this implementation - no > complaints. Things could be optimised here and there (e.g. event driven > COVs rather than an 'idle loop'), but for what it is - primarily a > simple-as-possible implementation/starting point - it is just fine. > > And brace yourself - you are likely in for a $10K++ bill from BTL, even > for > a relatively simple device. > > Ed > ***@***.*** > www.bac-test.com > > > > On Sat, Jan 20, 2024 at 2:37 AM josephtoutoo ***@***.***> > wrote: > > > Is anyone did BTL Certification using this stack. if you have > information. > > Could you please provide information. > > > > — > > Reply to this email directly, view it on GitHub > > < > #515 (comment)>, > > > or unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AAEBER7NZMQB4WFBKXRRR2TYPMUXFAVCNFSM6AAAAAA6E6B7N6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBRGY2DGNJVGU> > > > . > > You are receiving this because you commented.Message ID: > > ***@***.***> > > > > > -- > > > > *Edward Hague**BACnet Interoperability Testing Services, Inc* > +1 (650) 305 1395 > 155 Fleet Street, Portsmouth, NH 03801 > www.bac-test.com > LinkedIn: http://www.linkedin.com/in/edwardhague > > — > Reply to this email directly, view it on GitHub > < #515 (comment)>, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AEVJ36MB5YCNNSNMZSOC3HLYPUWW3AVCNFSM6AAAAAA6E6B7N6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBSGY3DQNJSGA> > . > You are receiving this because you commented.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#515 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEBER4KJW46NYRBLGV5A4TYPUYQ7AVCNFSM6AAAAAA6E6B7N6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBSGY3TENJVGU> . You are receiving this because you commented.Message ID: ***@***.***>

-- *Edward Hague**BACnet Interoperability Testing Services, Inc* +1 (650) 305 1395 155 Fleet Street, Portsmouth, NH 03801 www.bac-test.com LinkedIn: http://www.linkedin.com/in/edwardhague

from bacnet-stack.

skarg commented on July 22, 2024

The cyclomatic complexity reports can be generated via the Makefile (make pmccabe, or make complexity) which show the most complex functions:

$ make complexity

Complexity Scores
Score | ln-ct | nc-lns| file-name(line): proc-name
   32     128     118   ./src/bacnet/getevent.c(185): getevent_ack_decode_service_request
   32     221     203   ./src/bacnet/bacapp.c(601): bacapp_context_tag_type
   35      55      43   ./src/bacnet/basic/service/h_wpm.c(43): write_property_multiple_decode
   36      48      44   ./src/bacnet/basic/binding/address.c(195): address_file_init
   37     163     157   ./src/bacnet/bacapp.c(2417): bacapp_parse_application_data
   41     152      95   ./src/bacnet/npdu.c(370): bacnet_npdu_decode
   41     220     207   ./src/bacnet/lighting.c(151): lighting_command_decode
   45     163     148   ./src/bacnet/basic/service/h_apdu.c(544): apdu_handler
   48     171     147   ./src/bacnet/readrange.c(148): rr_decode_service_request
   54     140     121   ./src/bacnet/basic/service/h_getevent.c(63): handler_get_event_information
   59     257     245   ./src/bacnet/lighting.c(1126): color_command_decode
   97     162     144   ./src/bacnet/basic/service/h_rpm_a.c(41): rpm_ack_decode_service_request
  111     150     136   ./src/bacnet/proplist.c(94): property_list_encode
  116     411     315   ./src/bacnet/event.c(135): event_notify_encode_service_request
  154     358     262   ./src/bacnet/bacdest.c(614): bacnet_destination_from_ascii
  183     432     418   ./src/bacnet/bacapp.c(1735): bacapp_snprintf_value
  628     311     240   ./src/bacnet/basic/service/h_rpm.c(174): handler_read_property_multiple
  668     651     575   ./src/bacnet/event.c(551): event_notify_decode_service_request
total nc-lns     3618

$ make pmccabe

Score | function
99 ./src/bacnet/event.c(558): event_notify_decode_service_request
89 ./src/bacnet/bacapp.c(1964): bacapp_snprintf_value
71 ./src/bacnet/lighting.c(1134): color_command_decode
67 ./src/bacnet/bacapp.c(1266): bacapp_decode_known_property
66 ./src/bacnet/bacapp.c(1122): bacapp_known_property_tag
63 ./src/bacnet/property.c(1306): property_list_required
63 ./src/bacnet/property.c(1098): property_list_optional
62 ./src/bacnet/bacapp.c(766): bacapp_context_tag_type
58 ./src/bacnet/lighting.c(159): lighting_command_decode
45 ./src/bacnet/bacdest.c(628): bacnet_destination_from_ascii
43 ./src/bacnet/bacapp.c(2708): bacapp_parse_application_data
42 ./src/bacnet/lighting.c(553): lighting_command_from_ascii
40 ./src/bacnet/bacapp.c(3202): bacapp_same_value
39 ./src/bacnet/basic/service/h_apdu.c(557): apdu_handler
38 ./src/bacnet/proplist.c(98): property_list_encode
38 ./src/bacnet/basic/service/h_rpm.c(192): handler_read_property_multiple
33 ./src/bacnet/npdu.c(374): bacnet_npdu_decode
31 ./src/bacnet/list_element.c(96): list_element_decode_service_request
31 ./src/bacnet/bacapp.c(239): bacapp_decode_data
30 ./src/bacnet/readrange.c(151): rr_decode_service_request

from bacnet-stack.

Bacnet testing with BTS about bacnet-stack HOT 8 OPEN

Comments (8)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent