A search for pair production of up-type vector-like quarks ($T$) with asignificant branching ratio into a top quark and either a Standard Model Higgsboson or a $Z$ boson is presented. The same analysis is also used to search forfour-top-quark production in several new physics scenarios. The search is basedon a dataset of $pp$ collisions at $\sqrt{s}=13$ TeV recorded in 2015 and 2016with the ATLAS detector at the CERN Large Hadron Collider and corresponds to anintegrated luminosity of 36.1 fb$^{-1}$. Data are analysed in the lepton+jetsfinal state, characterised by an isolated electron or muon with high transversemomentum, large missing transverse momentum and multiple jets, as well as thejets+$E_{T}^{miss}$ final state, characterised by multiple jets and largemissing transverse momentum. The search exploits the high multiplicity of jetsidentified as originating from $b$-quarks, and the presence of boosted,hadronically decaying top quarks and Higgs bosons reconstructed as large-radiusjets, characteristic of signal events. No significant excess above the StandardModel expectation is observed, and 95% CL upper limits are set on theproduction cross sections for the different signal processes considered. Thesecross-section limits are used to derive lower limits on the mass of avector-like $T$ quark under several branching ratio hypotheses assumingcontributions from $T \rightarrow Wb$, $Zt$, $Ht$ decays. The 95% CL observedlower limits on the $T$ quark mass range between 0.99 TeV and 1.43 TeV for allpossible values of the branching ratios into the three decay modes considered,significantly extending the reach beyond that of previous searches.Additionally, upper limits on anomalous four-top-quark production are set inthe context of an effective field theory model, as well as in an universalextra dimensions model.