Militia in Bury

While chatting to a researcher in the Fusiliers Museum he told me that there had once been a barracks and parade ground in Silver St but he didn't know it's exact whereabouts, which may have been straddling Broad St which at that time didn't yet exist. Since hearing this I've read an account of someone remembering that the Militia were at one time billeted on Kay Gardens which would be before the Market moved onto the site in the 1800s. The old Militia Barracks which stood on Bolton Rd at the end of Stephens St was opened in the 1850s so all this must have been before then. Anybody know anything about it ?