backup power equipment failures 'brought gnb down' - battery power storage

Prime Minister Brian Gallant said he would "get to the bottom" of what happened to Juni, when the government server experienced two serious crashes one morning after a power backup device failure at the marisville data centerMDC).
MDC has IT systems used in justice, health, public safety, social development, finance, etc.
Great impact
Arrived, took a few days to repair and even asked for participation at the time
Prime Minister David Ward
"Our government is seriously responsible for protecting the privacy and personal data of New Brunswick people," Gallant said in a statement on Tuesday . " This is a response to the CBC News survey.
Gallant said the government would "take steps to ensure that this does not happen again ".
He said the external review initiated by the former government is already under way.
"We intend to publish the findings and suggestions as timely as possible.
"These events began on the morning of June 9.
A Osprey is nesting on the NB transmission line.
This bird is one of hundreds of birds doing the same thing in the province's grid every spring, and somehow short-circuited electricity during its construction.
Customers in the greater fredricton area, including the marrisville data center, have been cut off.
The first warning was at 9: 27 A. M. m.
: "We are currently using battery power," wrote Trevor McDonald of the in-house service (NBISA).
The data center downtime affected some computer services in the judicial system and the court's decision was put on hold due to documents that were inaccessible. . . [and]
Maybe someone is sitting [cells]
Well, this may not be true.
"Because the judges were unable to access computer documents, the court ruling involving some shale gas protesters was postponed.
The main systems in the health and motor vehicle sector are faulty.
Concern about health
The nursing staff of the Ministry of Health's mental health and addiction mobile crisis unit, during a planned outage, were in critical circumstances unable to access the customer's historical data center on the computer.
The New Brunswick service declined those who wanted to renew their driver's license.
Its website is also offline.
Car dealers are unable to complete the online paperwork on the New Brunswick system. Some civil servants are unable to work for hours or even days.
In some cases, overtime is required to clear the backlog.
NB wine can't handle credit card ordebit card transactions in the store, the time for its retail business to "basically Drive" "If the power supply doesn't pay off for the next month-
After 20 minutes, the system will have a hard shutdown.
"There is a problem.
Automatic switch (ATS)
When the street is powered off, the data center should be connected to a diesel generator but not working.
When technicians rush to manually switch, the data center is running on a giant battery pack called uninterrupted power supply (UPS).
But the battery was exhausted before the backup diesel generator powered the data center.
"The mainframe NB government has gracefully shut down the UPS battery before it reaches the threshold to support the load, and all other systems are seriously faulty," wrote a Bell aliantooperations manager at the data center . ".
Switch to diesel power but hard in the data center after two hours-
Crashed again when the technician tried to repair the ATS.
The host is also broken this time. 'Yes. No. F--
Kit is badit employees work all day long and early morning the next day to restore critical systems, corporate services and restore public services
After the backup power is restored, face the website.
Internal Service personnel engaged in recovery work described the situation in a colorful language in the letter. “Yes. No. F--k it is bad.
When asked if there was an estimated arrival time for recovery, an employee replied.
Another wrote: "Yes . . . . . . The entire data center is basically vomiting. Power issues.
When someone asks them to lose contact with various programs.
In the public, comments are also flying around.
Melanie Morris wrote on Twitter, "the day I'm on vacation this week, I'm going to update my license and the New Brunswick system is down.
Hope I won't be pulled over!
"She told CBC News that the staff who served New Brunswick politely told her that they could not update her license because all their systems were closed.
"In order to get there and find that nothing is working, I'm a little frustrated by turning around and coming out.
She added that she returned on the following Saturday and updated her license.
"There are a lot of things that end up being postponed," said criminal defense lawyer Alison Maynard . ".
In June 19, Maynard's client, Jayman junille Braun, from Lexington. B.
And Aaron Francis from Eskasoni, NY. S.
He was detained after admitting some allegations related to the violent fracking protests in October. 17, 2013.
The couple are waiting for a decision on the other allegations they rebutted in relation to the violence, but the decision has been postponed because Judge Leslie Jackson was unable to access the key documents in the case.
"It will certainly be frustrating for those who are imprisoned," Maynard said . ".
In June 11, a technician was assigned to assist specifically in addressing the problems that hindered the work of the justice system.
An email commented: "Yes, the situation is getting worse.
Because of the files they can't access, they have an uncertain decision . . . . . . So people might sit there instead of sitting there.
I was wondering if they thought they should restore WHO's network drive first. ”The hard-
In some departments, the crash caused a chain reaction. Data-
The corruption of the data storage system has caused problems that some departments need a few days to recover.
Christian Couturier, chief information officer of the province, said in an email on June 10: "The bigger problem is that data on the storage array may be destroyed.
This is being turned off as the main system (
In health and motor vehicles).
The shared service organization is working hard to solve (
Recognition and Restoration of data. Rough day here.
ANBISA director Angie Milbury, an email dated June 17, disclaimer "The purpose of this message is to explain to the IT supervisor the method we have taken to ensure that all GNB file server data affected by the interruption is restored.
"In order to effectively handle the massive recovery caused by downtime, we are finalizing a plan to restore file server data from the backup on June 6 . . . . . . " It continued.
'You are playing with fire' on the morning of June 9, NB Power resumed Power within an hour, but during the first crash, UPS was "bombed.
The email with the theme "fuse" and the attached picture said, "That's why GNB was knocked down.
"The record shows that it was soon learned that at 25-year-old unit.
MDC cannot switch back to street power without UPS.
The company has been using diesel for two weeks.
The email states that fuel consumption ranges from 50 to 85 litres per hour.
The morning June9 further complicated the problem, after connecting the diesel power to the MDC, the generator began to have problems keeping the frequency, prompting the rush to find a portable generator to replace the usual redundancy.
It is not clear from the record what caused a backup power system failure.
Perform regular maintenance and testing of the system.
In fact, UPS performed maintenance the day before the power outage. “. . .
If there is only one generator [as a back up]
Then you're not redundant.
So basically you're playing with fire, "Berdini said.
"Nothing is 100
But there are ways to maximize uptime.
This is the basic method.
If you know what you're doing, it's very easy to guarantee uptime.
But again, it's money.
If you don't have the money to have twice the equipment then you get stuck.
This may be a matter of money.
Berdini is not the only one who wants to know what is behind failure.
Finance and productivity had a significant impact within 10 days of the crash
Prime Minister David Alward wrote to Mark Leger, the province's top bureaucrat, that he is the chairman and cabinet secretary of the executive committee.
Alward askedler will lead a committee to review technical glitches and find out how to avoid similar things happening in the future.
"As you know, the failure of the marisville square, which started in June 9, 2014, has had a significant financial and productivity impact on the New Brunswick government.
"I know this has had a negative impact on the services of citizens and businesses and has disrupted the offices of the entire network," Alward said . ".
After the Osprey incident, are you affected by the loss of computer services in the provincial government
Related power outage?
Contact us at atcbcnb @ cbc.
Cato, tell us your story.
In addition to the clerk's committee, Auditor General Kim McPherson is also reviewing the power outage to determine whether this should be the subject of her next report.
CBC News also learned that Ernst & Young was commissioned to study the blackout and will submit a draft report to the secretarial committee on Monday, but the government neither confirmed nor denied the facts.
CBC News asked for research interviews and
Camera interviews and sent dozens of questions to seven different government departments asking about the impact of the power outage, trying to learn more about what the real impact is and how long they have been, how to solve it, and how much the power outage costs taxpayers
For replacement parts, overtime and productivity downtime.
No questions were answered and all interviews were rejected.
Government service spokesman Sarah Ketcheson wrote in a statement, "an accident review is currently under way for a power outage in June 9.
This is a standard procedure when an event occurs that has an impact on our IT system.
Once the review is completed, the government has the right time to study the final report and we will be able to answer your questions better.
"On June 20, two days after the prime minister's letter was sent, nearly two weeks after the initial power outage, the diesel generator that powered the MDC failed. An 8:18 a. m.
An email from an NBISA employee said that most services were backed up again after the "portable generator failed this morning (
Apparently smoke or something).
Two days later, a temporary UPS was placed in a tractor.
A Penn company's trailer, which rents about $52,000 a month, is connected and allows the MDC to reconnect with street power.