Test and Certify with the SmartThings Test Suite

The SmartThings Test Suite, part of the SmartThings Console, automates Works with SmartThings (WWST) product testing, ensuring device compatibility with the SmartThings platform.

In many cases, completing the required tests may qualify your products for no-cost, self-test certification, eliminating the need to ship your devices to our Authorized Test Provider. Note that while the Test Suite supports testing of many device connection protocols, only Matter, Zigbee, Z-Wave, and Cloud-to-Cloud (Schema App) protocols are eligible for self-test certification at this time.

⚙️ Supported Protocols for Device Testing

The Test Suite supports device testing for the following integration types:

• Matter
• Zigbee
• Z-Wave
• Cloud-to-Cloud (Schema App)
• LAN
• MQTT

Note that not all device integration types supported by the Test Suite qualify for self-test certification.

⚡️ Qualify for Self-Test Certification

If your device meets all of the following criteria, your device may qualify for self-test certification, eliminating the need to ship your device to an Authorized Test Provider:

• Connects via Matter, Zigbee, Z-Wave,
  Cloud-to-Cloud (Schema App), or MQTT (Direct Connected)
• Aligns with a Device Type supported by the Test Suite
• Contains only Capabilities supported by the Test Suite
• Has completed a Full Integration Test in the Test Suite.

Some device types (e.g., cameras, doorbells, smart locks, and health/safety products) may not qualify for self-certification and will require evaluation by our Authorized Test Provider regardless of protocol.

Z-Wave + SmartThings Hub

If you are using a SmartThings Hub to connect with your Z-Wave device, the following Z-Wave Plus V2 features are not supported by the SmartThings Hub:

• Z-Wave Long Range star network
• SmartStart onboarding
• Over-the-Air (OTA) updates

Prerequisites

Before using the Test Suite, be sure you have:

1. A Samsung account.
2. The latest version of the SmartThings app installed on your iOS or Android device.
3. A device connected to the SmartThings platform using Matter, Zigbee, Z-Wave, LAN, MQTT, or a Schema App (note that while these integration types may be tested using the Test Suite, not all integration types are supported for self-test certification).
4. If testing a device that is part of a Developer Center Organization, ensure that your Samsung account is associated with the Organization. Visit Using the SmartThings Console - Organizations to learn about Organizations and how to invite others to an Organization.
5. Reviewed the device types and Capabilities supported by the Test Suite. Note that not all device connection protocols supported by the Test Suite qualify for self-test certification.
6. If you are completing a Full Integration Test (required for Self-Test Certification), ensure your device is not currently onboarded. You will be prompted to onboard the device during the testing process.

Get Started with the Test Suite

To get started, sign in to the Test Suite using your Samsung account.

The Test Suite will display a list of all devices currently associated with your Samsung account. The Test Suite device list page is your central hub for managing and organizing devices in the Test Suite, including:

• Connected Devices: All devices linked to your SmartThings account.
• Shared Devices: Devices available through Locations and Organizations shared with your SmartThings account.
• Previously Connected Devices: Devices no longer connected but have completed tests in the past to ensure you have complete history available.
• Device Test History: Test status and history for devices associated with your Samsung account.

Missing a shared device?

If you are not seeing an expected device, visit my.smartthings.com/advanced/devices to confirm your device is visible in your Samsung account. If not visible, ensure your Samsung account is associated with the shared Location and/or Organization. For further assistance, email support.

Start a Test Session

The Test Suite supports two types of tests:

• Full Integration Tests: A comprehensive test that verifies device onboarding, functionality, and removal in accordance with WWST standards. Completing a full integration test is required for your device to be eligible for self-test certification.
• Operational Tests: Intended to assist during the integration development process. Operational tests enable you to individually test aspects of a full integration test, such as device functionality or onboarding. Completing only operational tests does not automatically qualify your device for self-test certification.

Select New test next to your desired device to start the test session creator. If you are conducting a full integration test, remember that your device must not currently be onboarded to the SmartThings platform - device onboarding is part of the full integration test process.

If your device is not yet associated with your Samsung account, click Start Full Integration Test on the Test Suite landing page to begin an integration test and start the device onboarding process.

Onboarding

The full integration test starts with onboarding your device to the SmartThings platform. Before starting the test, you will be prompted to select your device connection type:

Hub Connected Devices (Matter | Zigbee | Z-Wave)

1. Prep Your Device
Please make sure that your device is ready to be added to the SmartThings platform. If the device you wish to test is already connected, please remove it from the SmartThings platform before proceeding.

2. Device Detection
Once your device is ready to be onboarded, the Test Suite will detect and display newly onboarded devices. All found newly onboarded devices will be displayed; select the desired device to continue the test.

3. Confirm Onboarding Method
Select the method used to onboard your device to SmartThings.

Cloud Connected (Schema) Devices

Prerequisites

Before beginning, ensure that you complete the following requirements:

• Remove linked services: Remove any previously linked services (Schema connectors) from SmartThings that you intend to test. For instructions, see the SmartThings Support Guide.
• Prepare test devices: Have two test devices available for testing:
  • One primary device to test core functionality.
  • One secondary device for additional onboarding verification.
• Gather required documentation: As part of the test process, you will need to upload the following documentation during the test:
  • A screenshot demonstrating correct implementation of the authorization page.
    • Accepted formats: PNG, JPG, JPEG
    • Maximum file size: 25 MB
  • A video demonstrating App-to-App Account Linking (if your integration supports this feature) on applicable systems (Android, iOS).
    • Accepted formats: MP4, MKV, WebM, MOV, WMV
    • Maximum file size: 100 MB

Required Screenshot: Authorization Page

Note: Have your screen capture tool ready. You must capture a screenshot of the authorization or login screen during the final re-pairing phase.

Preparation

1. If App-to-App Account Linking is supported, uninstall your app.

Pairing process

1. Initiate the first-time device pairing process with SmartThings.
2. Observe redirection to your web login page.
3. Enter your credentials.
4. Confirm successful device pairing in SmartThings.

Service removal

1. Remove the linked service from the SmartThings app.
2. Verify that the service has been completely removed.

Re-pairing process

1. Re-pair the device shortly after removal (while the authentication data is still cached).
2. Observe the display of the authorization screen or the web login page.
3. Capture a screenshot. You will upload this screenshot to the Test Suite when completing a full integration test.

Required Video: App-to-App Account Linking (if applicable to your integration)

Preparation

1. Install and set up your app.
2. Log in and add your test device.

Action Required: Begin your screen recording now. Ensure the capture is active before proceeding to the pairing process.

Pairing process

1. Launch pairing via the SmartThings app.
2. Confirm automatic redirection to your app.
3. Approve the connection via the authorization screen (without re-entering credentials).
4. Confirm successful device pairing in SmartThings.

Service removal

1. Remove the linked service from the SmartThings app.
2. Verify that the service has been completely removed.

Re-authentication test

1. Log out from your app.
2. Launch pairing again via the SmartThings app.
3. Confirm that you are redirected to your app and prompted to log in.
4. Complete the login and pairing processes.
5. Verify the successful connection of the device.

End your screen recording now.

Device detection

After gathering any required documentation, you are ready to proceed with the test. The test will start with the onboarding process. The Test Suite will detect and display any newly onboarded devices.

Ensure your device has been removed from your Schema app before proceeding with the onboarding test - the detection process will not identify any devices that have already been connected through your Schema app.

Any new devices that are added as a result of new integrations will be displayed. Choose the one you would like to certify.

Onboarding method and supporting documentation

Select the method you used to add your device to SmartThings. Upload any required documentation (screenshot, video) as described in the prerequisites.

Direct Connected Devices (MQTT)

Prerequisites

Before beginning, ensure that you complete the following requirements:

• Complete security requirements: Review and complete the security requirements for direct connected devices.
• Gather required documentation: As part of the test process, you will need to upload the following documentation during the test:
  • A photo of the device's QR code, corresponding to the device to be onboarded (required even if the QR code will not be used for onboarding).
    • Accepted formats: PNG, JPG, JPEG
    • Maximum file size: 25 MB
    • The QR code must follow the Direct Connected Devices QR code guidelines.

Device Functionality Testing

The Test Suite will analyze your device and generate a customized test plan.

You can exclude specific Capabilities, Commands, or Attributes from testing (useful if you are refining your integration and want to focus on specific elements). Test cases will run in the order in which they are displayed on the page.

Certain Capabilities may automatically be excluded from testing. Any automatically excluded Capabilities are listed in the Not Supported in Test Suite section. If your device contains automatically excluded Capabilities, you may still submit your device for WWST certification; we will evaluate your submission to determine if additional testing is required.

For WWST certification, we recommend including all elements in your test session

If exclusions are necessary, provide a detailed explanation in the "Notes for SmartThings Team" field on the Test Summary page.

Types of Test Cases

1. Fully Automated Test Cases: These cases run automatically and without user intervention; simply monitor progress. For example, an RGBW lightbulb will automatically turn on, dim, and change color. The Test Suite will monitor to confirm all events have occurred as expected

2. Manual Action Test Cases: These test cases require specific manual interaction within a limited time frame. You will be asked to physically interact with the device to trigger an event, such as motion detection, contact sensor activation, or a temperature change. You can use the Extend button to request additional time to complete the test. The availability of this feature and the amount of permitted extensions depend on the individual test case.

3. User Feedback Test Cases: These test cases require specific manual interaction within a limited time frame. You will be asked to note the device's physical properties or changes to these properties. For example, the tool will trigger a sound on the device and ask you to confirm when you hear the sound. You can use the Extend button to request additional time to complete the test. The availability of this feature and the amount of permitted extensions depend on the individual test case.

After reviewing your device test plan, click Start Test Session to begin testing.

Monitor Your Test Session

The Test Session in Progress page provides real-time monitoring of test case execution.

Skipping Manual Operations

Manual actions or user feedback can be skipped, even if these actions were selected during setup. If you are conducting full integration testing, we recommend completing all test cases for full compliance.

Review Your Test Results

After all test cases are completed, you’ll be redirected to the Test Summary page.

The Test Summary provides a detailed overview of your test session, including results from all executed test cases.

• View Test Case Details: Review pass/fail statuses and any issues encountered.
• Download Your Test Report: Download your test report, including detailed logs for deeper analysis and troubleshooting.
• Share Logs With Support: Share logs with the SmartThings support team to investigate major issues.

If your device does not fully pass all tests, WWST certification may still be possible

If you think your device may still qualify for certification, describe your case in the “Notes to the SmartThings Team” section in your certification submission form.

Link Your Test Results With Your Product and Submit for Certification

If your device passes all tests required by the Test Suite and meets the qualifications for self-test certification, your device may now be submitted for Works with SmartThings certification. In order to certify based on Test Suite results, you must link your test results with the product you are certifying.

tip

Before you can link your test results with a product, you must first create a product in the SmartThings Console.

To link your test results with your product:

1. Expand the Test history of your device.
2. Select Link to product and select the desired product to associate with the specified device test history.
3. Submit your product for certification in the SmartThings Console. We recommend reviewing the certification guidelines before submitting your request.

After you submit your certification request, SmartThings will review your request and contact you with your next steps. You can read about the certification review process in-depth in Apply for Certification.

Supported Device Types

The list below details every device type currently supported for self-test certification in the Test Suite.

To be eligible for self-test certification, ensure that your device:

• Connects via Matter, Zigbee, Z-Wave, Cloud-to-Cloud (Schema App), or MQTT (Direct Connected Devices)
• Aligns with a Device Type supported by the Test Suite
• Contains only Capabilities supported by the Test Suite

Supported Device Types
AirPurifier
AirpurifyHumidifier
AirQualityDetector
BathroomHeater

Supported Capabilities

Your supported device type must also include only Capabilities that are currently supported by the Test Suite. You can find a comprehensive list below:

Supported Device Capabilities
accelerationSensor
activityLightingMode
activitySensor
airConditionerFanMode