Article contents 17 sections
A smart lock pilot is a small installation on chosen project doors that answers one question: can the rest of the project go ahead with this lock, this installation method and this way of running it? To plan one, pick doors that cover the real differences on site, define the tests and the pass criteria first, run real users through real workflows, and decide in advance what result means expand, fix and retest, or stop.
A lock that opens on a demo stand has not answered that question. On site, the result also depends on the door and frame, the mounting parts, the users, power, connectivity and the people who manage and maintain it.
The project requirements and quotation that come before a pilot are covered in the smart lock project specification and RFQ guide.
What Should a Smart Lock Pilot Prove?
A pilot answers specific deployment questions. Can the proposed assembly be installed without unapproved alterations? Does the door secure and release through the intended sequence? Can ordinary users operate it reliably? Can the administrator enroll and revoke access? Can the site team handle a problem without relying on undocumented assistance?
A short pilot can reveal field issues, but it does not replace laboratory testing: a few weeks on site will not establish battery life over years, all-season outdoor durability or a biometric error rate. Factory inspection is also separate: it checks the shipped goods against the order, while the pilot checks the installed configuration on the site.
The result names which configuration worked, on which openings, under which conditions, and what must be resolved before the next group is installed.
1. Define the Decision and Scope Before Choosing Doors
Write a short pilot brief: the planned rollout and the open questions the pilot must close, each concrete enough to give an observable answer.
“Evaluate smart locks” is too broad. “Confirm installation on the existing narrow-stile doors without changing the approved preparation” is a specific question. So is “confirm that the site administrator can revoke a staff credential using the intended management process.”
The brief states:
- The proposed configuration and opening groups being evaluated.
- The required operating functions and user workflows.
- The uncertainties that prevent a larger deployment decision.
- Who supplies, installs, observes and reviews the pilot.
- The planned tests, observation coverage and acceptance criteria.
- The recovery arrangement if an opening cannot be used normally.
- Whether pilot units count toward the final order.
Define the possible outcomes in advance: expand, correct and retest, change the configuration, or stop. Test optional features only when the result will change the order or the rollout plan; otherwise they just delay the decision on the functions the project needs.
2. Select Representative Openings, Not Just Convenient Ones
Choose doors by the differences that change installation or operation. Four doors that each represent a different condition tell you more than six identical doors in one corridor.
| Selection factor | What to include where relevant | What an easy demonstration misses |
|---|---|---|
| Door construction and preparation | Different materials, profiles, thicknesses and existing cutouts | Mounting or lock-body incompatibility |
| Handing and leaf arrangement | Relevant opening directions and active/inactive leaf arrangements | Reversal, clearance or coordination problems |
| Frame and receiving hardware | Representative strike positions, gaps, closers and seals | Binding or unreliable engagement |
| User workflow | Routine users, occasional users and administrators | Confusing actions or unworkable access management |
| Traffic pattern | Typical use and relevant busy periods | Queueing, repeated attempts or closing behavior |
| Connectivity | Actual gateway locations, walls and network arrangement | Dependence on a temporary nearby connection |
| Exposure | Relevant sheltered or exposed locations within the product's intended use | Conditions absent from an indoor trial |
| Service access | Representative battery, cable and component access | Maintenance that requires unexpected dismantling |
A visibly defective door is a poor test door: repair it first, or the alignment problem will hide the question you are trying to answer.
Each distinct configuration needs its own door. A local-only lock that works says nothing about the remote-managed version, and a battery lock says nothing about a wired one.
How Many Pilot Doors Are Enough?
Enough to cover each important variation, often a handful of doors, as in the four-door example below; there is no fixed percentage. One door can cover several factors at once, such as a thickness, a handing and a busy traffic pattern. A different installation method or configuration still needs its own door.
A small pilot checks coverage. It is not a reliability study, so a few good installations do not give a failure rate for the whole project.
3. Survey the Selected Doors Before Installation
Record each opening's identity and existing condition before removing hardware. Capture the door faces, edge preparation, frame, receiving hardware and relevant cable routes. Record dimensions with clear reference points and retain the survey date.
Check that the door closes and latches normally with its existing hardware, and note dragging, hinge wear, frame movement, seal pressure and damage. That record separates old problems from new ones.
The Steel Door Institute lists alignment, hinges and the strike among the causes of binding in steel openings, which is why the door and frame are checked before a problem is blamed on the electronic lock. SDI installation troubleshooting guide
Compare the selected assembly with the surveyed opening. Identify the intended mounting kit, lock body, spindle, strike and preparation drawing. Resolve conflicts before cutting or drilling.
For fire-rated or otherwise restricted openings, get the required review and the permitted work agreed before installation. UL notes that field changes to rated doors can affect their listing and need evaluation; a working pilot does not replace that review. UL doors and related hardware application guide
4. Record the Configuration Actually Being Tested
Identify the hardware and software configuration installed at each pilot opening. Include the lock assembly, lock body, mounting parts, power source and relevant firmware, app or gateway information where available.
Record the operating settings too, such as auto-relock, privacy mode and credential permissions; two identical locks behave differently with different settings.
Use the project's own equipment. A temporary gateway placed beside the door or a demo account with wider permissions gives results that will not carry over. Where a stand-in is unavoidable, write down the difference and what still has to be tested on the final network.
Log changes as they happen. If a new spindle, a firmware update or a revised strike fixes a problem, the approval covers that revised configuration, and the original failure stays in the record.
5. Assign People, Access Windows and Recovery Arrangements
Name the trial coordinator, the installer, our technical contact, the network or platform contact where there is one, the site administrator and the person who decides the next step. On a small project one person can hold several roles. Establish who records an issue, who can change settings and who must be contacted if the door cannot be secured or used normally.
Agree installation and testing windows with the site. Account for occupied rooms, cleaning, deliveries and the need to keep normal access available. Brief participating users on what is changing and how to report difficulty.
Plan recovery before the old hardware comes off. Keep the original parts if you may refit them, and if the new preparation cannot be reversed, agree another fallback first.
Give authorized staff a way to open or secure each pilot door. A mechanical key solves an electronic fault but not a jammed mechanism, so the response plan covers both.
6. Define Acceptance Criteria and Stop Conditions
Keep the must-pass requirements apart from the observations that improve the rollout. Inside operation, correct locking and authorized access are not averaged with cosmetic preferences or training comments.
| Area | Example acceptance basis | Response to an unresolved problem |
|---|---|---|
| Installation | Assembly fits using the accepted parts and permitted preparation | Review the interface or installation method before expansion |
| Mechanical operation | Door closes, secures and releases as specified under normal site use | Investigate door, frame and mechanism conditions |
| Access control | Valid and revoked credentials behave according to the agreed rules | Keep the logs and correct before relying on the affected function |
| Interruption and recovery | Required behavior is demonstrated for the applicable power or connection scenario | Identify the failing dependency and repeat the relevant tests |
| Administration | Intended site personnel can complete the required management tasks | Correct permissions, process or support arrangements |
| User operation | Intended users can complete the normal sequence with the agreed instruction | Distinguish usability, training and product issues |
| Deployment readiness | Accepted configuration, instructions and unresolved exceptions are recorded | Limit expansion to the scope the pilot covered |
Set numerical criteria only where the project needs them and the measurement method is defined. For response time, identify the starting event and the physical completion event. For installation time, distinguish preparation, fitting, configuration, testing and repair of existing conditions.
Stop the affected activity if it creates an unsafe condition, unexpected unauthorized access, loss of required inside operation or a door that cannot be secured as intended. Restore an acceptable condition through the agreed response plan before continuing.
Use four results: pass, fail, not tested and not applicable. A missing record is "not tested", not a pass.
7. Install and Check the Complete Opening
Follow the applicable installation instructions and accepted preparation drawing. Record the parts actually used and any deviation from the intended method.
Check operation before engaging the receiving hardware, then with the door closed in its normal condition. Observe the effect of final fastening, seals, closer action and the strike. A lock that works only when mounting screws are loosened has not demonstrated the intended installation.
Record installation time by category. The first door often includes training or a door repair that will not repeat, and a factory technician's fast install is not the site crew's pace.
The door-hardware guide specification UFGS 08 71 00 treats installation and field quality control as a stage separate from the product requirements, which is a good model for the pilot record. UFGS 08 71 00
Before leaving the opening for routine use, check the intended entry and inside operation, normal closing and locking, and the recovery method. Record any temporary settings so they cannot be mistaken for the final operating configuration.
8. Test Real Workflows and Relevant Failure Scenarios
Normal Access and Locking
Have users perform the actual sequence: present the credential, complete any handle action, pass through, close the door and confirm the expected secured state. Test both entry and inside operation as required by the project.
Watch the door, not the screen. A reader beep or an app confirmation is not a released latch, and a closed-door indication is not a deadbolt in the strike.
Test each access method the project will use; a card result says nothing about fingerprint or remote access. Keep results by method and door where they differ.
Enrollment, Revocation and Shift Changes
Ask the intended administrator to add a user, issue the relevant access, remove that access and verify the result through the documented process. Include scheduled access or temporary users when these are part of the purchase.
Note how changes reach the lock. A revocation entered in the app but not yet synced to an offline lock is a different case from a synced revocation the lock ignores.
Use test identities and collect only what the trial needs. Live PINs, credentials and personal access histories stay out of circulated issue reports.
Connectivity and Recovery
Test the actual connection path, then interrupt relevant links separately through an agreed method. Internet loss, gateway loss and a phone's local connection are different test conditions.
Check what still works, what stops and how the system reports it, then restore the link and watch the sync. Make sure a phone's own connection is not quietly bypassing the interruption. Arrange these tests with the network team so other building services are not disturbed.
Power and Backup Operation
Run low-power and power-loss checks with the method in the manual, and know beforehand whether this lock stays locked or releases. Then test the backup procedure from the position authorized staff will use. Where the lock has an emergency power port, follow the instructed procedure and note what the port does on this model, such as whether it still needs a valid credential.
A power-loss test is not a jam test. Never jam a live door as a test.
9. Observe Ordinary Use Long Enough to Cover the Intended Conditions
Run the pilot until it has covered the workflows that matter, not for a set number of days: arrival, departure and cleaning in an office, a tenant or guest turnover in a managed property, the relevant weather at a gate.
Planned tests cover conditions that rarely happen on their own. Ordinary use shows the retries, confusing prompts, closing habits and service calls a demo never does.
Record the conditions you actually had. A dry month says nothing about rain, and a mild week says nothing about winter; list what was not covered instead of stretching the conclusion. Battery readings over a few weeks can flag a problem but do not give a battery life.
Ask users specific questions: what action failed, what did the display show, was the door open or closed, did a retry work, and was assistance required? “Works well” and “sometimes slow” are starting points, not complete findings.
10. Measure Results Without Hiding Problems in Averages
Count attempts, first-attempt successes, retries and unresolved failures separately, and define what one attempt is before you start.
Suppose a trial records 100 valid-credential attempts: 96 succeed initially, three succeed only after a retry and one remains unsuccessful. First-attempt success is 96%, while eventual success within the recorded retry process is 99%. Reporting only 99% hides the extra actions and the remaining failure.
Neither percentage alone determines acceptance. Review the failure's consequence, concentration by opening and compliance with the agreed criteria. An unexpected unauthorized release cannot be offset by many successful routine operations.
Keep test counts apart from everyday reports. App logs miss some physical attempts, and no complaints is not the same as a count.
Report installation and support time the same way: one-time learning, repeatable work, door repairs and repeat visits. That plans the next phase better than one average install time.
11. Investigate, Correct and Retest with Traceable Evidence
Describe what happened before assigning a cause. “Valid card accepted; actuator audible; latch did not release with door closed” is more useful than “bad lock.” The observation can guide checks of power, installation, receiving hardware and mechanism behavior.
For each issue, record the opening, configuration, conditions, sequence, expected result, observed result and consequence. Add the correction and the retest result.
| Finding | Immediate investigation | What closes the issue |
|---|---|---|
| Release works with door open but fails when closed | Check alignment, receiving hardware and load under the accepted setup | Required operation demonstrated after the documented correction |
| Remote access fails at one location | Identify the actual connection route and compare site conditions | Successful operation and recovery on the intended network arrangement |
| Users repeatedly retry a valid credential | Check presentation, prompts, credential state and physical result | Cause addressed and the relevant users repeat the workflow successfully |
| Revoked access still operates | Confirm revocation delivery and inspect the actual permission state | Correct rejection demonstrated under the required conditions |
| A replacement part resolves the problem | Identify the old and new part and other affected openings | Revised configuration documented and applicable tests repeated |
Keep intermittent failures on the list even when the next attempt works. An unknown cause is reported as unknown; a fault that could not be reproduced once is not solved.
Change one thing at a time where you can. After a fix, repeat the failed case and everything the change could touch: a firmware update can affect more than the symptom, and a mounting correction means repeating the closing, locking and release checks.
A Worked Example: Selecting and Reviewing a Four-Door Pilot
The following example applies the method to the 24-opening office retrofit used in the RFQ guide.
The project includes 18 office openings and six store-room openings, using two mounting arrangements and the handing conditions recorded in its survey. The team selects four openings to cover these differences and a busier user workflow. Special openings outside this work package remain outside the trial.
| Selected opening | Reason for selection | Main question |
|---|---|---|
| D01 office | Common mounting arrangement and ordinary traffic | Can the regular installer reproduce the accepted installation? |
| D08 office | Different handing within the office group | Does the intended assembly and operation work in that orientation? |
| D19 store room | Second mounting arrangement | Are the kit and receiving parts correct for this group? |
| D23 store room | Second arrangement with busier access | Does ordinary use expose closing or workflow problems? |
The four kits are included in the final total of 24. Before wider release, 20 openings remain uninstalled. The pilot plan includes installation checks, valid and revoked staff-card access, inside operation, closing and locking, and the specified backup procedure. Remote management is outside the base requirement and receives no approval from this trial.
During the trial, D23 releases normally with the door open but intermittently fails with it closed. The team records the condition and investigates the receiving-hardware relationship. After an approved correction, it repeats the affected tests and observes the busy workflow again. The report retains the original failure and identifies the corrected installation condition.
With the other criteria met, the decision approves the reviewed configurations with the updated installation check. It does not approve other lock bodies, remote management or door groups that were not tested.
If D23's problem stays unexplained, the store-room group stays on hold. The office group can still go ahead if the reviewers show the fault involves nothing the office doors share.
Decide Whether to Expand the Deployment
Base the decision on coverage, results and unresolved consequences. The useful options are:
- Expand the reviewed scope: required tests and conditions are covered, the accepted configuration is identified, and remaining minor actions have defined owners.
- Correct and retest: the proposal remains viable but a requirement has not been demonstrated.
- Expand only a supported subset: the results show which groups are unaffected and which remain excluded.
- Change or stop the proposal: the configuration or operating arrangement cannot meet an essential requirement within the permitted scope.
Write the decision with its exact scope: door groups, configuration, installation method and any conditions. A must-have function that failed is a "correct and retest", not "approved with comments".
Keep checking in the next phase. Later units, workmanship and door conditions still have to match the approved reference, and a larger rollout brings network, admin and staffing load that four doors did not.
Close the Pilot and Prepare the Handover
The final report should include the selected openings, reasons for selection, installed configurations, test coverage, observation conditions, issues, corrections, retests and deployment decision. Identify what was not tested and what that limits.
Update installation instructions and the door schedule with accepted changes. Retain useful photographs of the final assembly and provide the site team with the operating and recovery information it needs.
Remove test credentials and any installer or factory access that should not stay, hand the admin account to its owner, and check that the final configuration still does what is required. Do not factory-reset a lock that has already been commissioned for the site.
NIST's IoT baseline covers authorized configuration changes and restoring a secure configuration; the cleanup and handover steps themselves come from the lock's manual and the project plan. NISTIR 8259A
Frequently Asked Questions
How many smart locks should a pilot include?
Enough to cover each important difference in door, configuration and workflow, which is usually a handful of doors. There is no fixed count or percentage; a project with similar doors needs fewer than a mixed one.
How long should a pilot run?
Until it has covered the operating periods and planned scenarios you defined, such as a full turnover cycle or a busy week. Days without real use add little, and no short trial gives seasonal durability or battery life.
Can a factory demonstration replace a pilot installation?
No. A factory demo shows the functions, but not your doors, frames, network and users. Use it to prepare the pilot.
Should the pilot use the final gateway and account arrangement?
Yes, wherever you can. Temporary equipment and demo permissions hide deployment problems. If you must use a stand-in, record the difference and repeat those checks on the final setup before approval.
Does every pilot problem require rejecting the product?
No. Many are fixed by an installation, configuration or training correction, then retested along with the related functions. Unexplained or recurring failures stay visible in the decision.
Can we approve some door groups while another group needs work?
Yes, when the fault involves nothing the approved groups share. Check the hardware, firmware and installation method they have in common before calling a problem local.
What should happen to pilot units after approval?
They follow the agreed supply plan, and usually stay as part of the order. Units that stay get their final configuration recorded, temporary access removed and an entry in the asset and maintenance records. Units that come out leave the doors restored and the trial records kept.
References
- Steel Door Institute — Installation Troubleshooting Guide for Standard Steel Doors & Frames. Door, frame and receiving-hardware conditions relevant to installed operation.
- UL Solutions — Doors, Windows and Related Hardware Application Guide. Application and field-evaluation considerations for doors and related hardware.
- UFGS 08 71 00 — Door Hardware. Project-editable door-hardware guide specification with installation and field quality-control provisions.
- NISTIR 8259A — IoT Device Cybersecurity Capability Core Baseline. Authorized configuration and secure restoration capability areas.