Thursday Cloud & Infrastructure sync

Person #35: Can you hear me? I think my audio's—hold on, let me just switch to the headset. Can you hear me now? Person #34: Loud and clear. Go ahead. Person #35: Good. Okay. So, look, I know this is the Thursday sync but it's Monday, so I'm going to keep it tight. We've got a lot to get through. I want to start with the Cloud Operations Center. The dashboard is showing green across the board for the last seventy-two hours. No critical incidents since Thursday. The SOC did have a false positive on Friday morning—someone flagged a spike in auth attempts on the ERP gateway but it turned out to be the Deloitte team running their regression testing. So, no action needed. But I've asked them to give us a heads-up next time they schedule anything that touches the production environment. Person #34: Good. That's the kind of thing we need to keep an eye on. Unplanned changes are the enemy. We talked about this at the last steering committee—the change freeze windows for the migration are non-negotiable. If Deloitte needs to run tests, they go through the standard change request process like everyone else. Person #35: Exactly. And that's actually my next point. The change advisory board met this morning and we've got a bit of a situation with Wave Three. The Department of Municipalities and Transport—DMT—they've submitted their cutover plan for the finance module. But they haven't completed the data cleansing exercise. Their vendor, the Indian vendor, they're telling us they need another three weeks. Which puts us past the September 15th deadline for the data center exit. Person #34: Three weeks? For data cleansing? What exactly is the holdup? Person #35: From what I understand, the legacy system has a lot of duplicate vendor records. Like, a lot. The data quality team found something like forty percent duplication. And some of the records go back to before the merger. So they're trying to figure out which entity actually owns the contract. It's a mess. Person #34: Yeah, that's—look, I've seen this before. When I was at Dubai Smart Government, we had exactly this problem with the Dubai Municipality migration. They had vendors in the system from 2005 that were still marked as active. We ended up having to do a manual review of over twelve thousand records. It took us six weeks, not three. Person #35: Right. So maybe we should be planning for that. The issue is, the data center exit date is fixed. We've got the lease ending on the old ADDA facility in Mussafah. If we don't have DMT out by then, we're paying double rent on a facility that's running at fifteen percent utilization. The Department of Finance is not going to approve that. It's in the budget. Person #34: What if we do a phased cutover? Move the finance module for the core entities first, leave DMT on the legacy system for another two months, and then do a separate migration wave for them? Person #35: We could, but that creates a parallel run situation. We'd have to maintain the integrations between the old ERP and the new one. And you know how the integration testing went with the first two waves. It's not pretty. We'd be creating more risk, not less. Person #34: Hmm. What does the DMT leadership say? Have you spoken to the Undersecretary directly? Person #35: I spoke to the Executive Director of IT at DMT yesterday. He's confident they'll be ready by the end of September. He said—and I'm quoting here—"the vendor has assured us that the data cleansing will be complete by the 20th." But we've heard that before. Remember Wave Two? The Department of Culture and Tourism said the same thing, and they missed by two weeks. Person #34: I remember. But at the end of the day, we need to be firm about the deadlines. The whole point of the merger was to get away from this culture of rolling dates. If we let DMT slide, every other entity in Wave Three is going to come back and ask for an extension. And then the whole sovereign cloud migration is off track. Person #35: Agreed. So my recommendation is to hold the line. I'm going to send a formal memo to DMT today, copied to the Chairman's office, stating that the cutover date is firm and that any delay will require a sign-off from ADEC. That should get their attention. Person #34: Good. Do it. And to be fair to them, let's also offer some support. Maybe we can send a couple of our data engineers over to help with the cleansing. Not to do the work, but to supervise and make sure it's done properly. That way we're being helpful, not just threatening. Person #35: Yeah, that's—actually, that's a good idea. We have two guys on the data platform team who did the same exercise for ADEK. They know the process. I'll talk to them and get them assigned. Person #34: How many entities are left in Wave Three after DMT? Person #35: After DMT, we've got the Department of Energy, the Abu Dhabi Media Office, the Judicial Department—no wait, ADJD is in Wave Four. Let me check my list. (pause) So, in Wave Three we have DMT, Energy, Media Office, the Abu Dhabi Pension Fund, and the Integrated Transport Centre. That's five entities. Four of them have already completed their data cleansing and are in the testing phase. Only DMT is the outlier. Person #34: Okay. So it's one entity holding up the whole wave. I'll bring this up at the Executive Council meeting on Wednesday. Make sure the Chairman is aware. I don't want him hearing about this from DMT first. That's the kind of thing that undermines trust. Person #35: Absolutely. I'll send you a one-pager tomorrow morning with the key facts—the dates, the duplication numbers, the vendor situation. You can use that if you need to brief him. Person #34: Appreciate that. Now, what about the TAMM side? Any issues with the new version that we need to be aware of from an infrastructure perspective? Person #35: TAMM 4.0 is stable. The AI assistant is handling about sixty percent of the traffic now. We did have a small spike in latency on Sunday night—nothing major, but the monitoring team flagged it. Turned out to be a batch job that was running longer than expected. The team at the Indian vendor, they're looking at optimizing the query that's causing it. Should be resolved by the end of the week. Person #34: Sixty percent? That's better than we projected. Do we have any numbers on the accuracy of the assistant's responses? Because the Customer Experience team is going to ask about that. Person #35: The latest report shows an eighty-seven percent accuracy rate. Which is good, but it means thirteen percent of the time it's giving wrong or incomplete information. And you know what happens when it gives wrong information—someone posts a screenshot on Twitter and we get a call from the Chairman's office. Person #34: Do we know what the failure modes are? Is it hallucinating fees again? Because that was the issue at GITEX when the demo showed a fee of two thousand dirhams for something that should have been free. Person #35: We've fixed that specific issue. The new knowledge base has been updated with the correct fee schedules from the Department of Finance. The remaining errors are mostly about eligibility criteria and document requirements. For example, the assistant might say that a document is required when it's actually optional. Small things, but they're still errors. Person #34: Can we put a mitigation in place? Like, if the assistant is not confident about an answer, it should say "I'm not sure, please contact the service center rather than continue to click on a document that's not required? Person #35: Yes. It's the standard "confidence threshold" approach. We can set it so that if the model's confidence is below a certain level, it defaults to a fallback response. We should have done this from the start, honestly. But there was pressure to launch at GITEX, and the team was focused on getting the features out. Person #34: That's the way it always goes. Launch first, fix later. But at some point, we need to be more disciplined about quality over speed. The government services are too important to have a thirteen percent error rate on the AI assistant. If someone uses TAMM to apply for a permit and gets the wrong information, it can cost them days. Person #35: I hear you. I'll make the confidence threshold a priority for this sprint. We should have it in place by the end of next week. And I'll also ask the team to do a deeper analysis of the error logs to see if there are any patterns we can address. Person #34: Good. Now, switching gears, what's the status of the Copilot rollout? We're supposed to be at thirty-five thousand civil servants by the end of the year. How are we tracking? Person #35: We're at about twenty-two thousand activated licenses. The issue is adoption. A lot of users are logging in once and then not coming back. We've seen this with collaboration tools before—people get excited at the training, but then they don't integrate it into their daily workflow. The Government Talent team is working on a change management program, but it's slow going. Person #34: Twenty-two thousand is good progress. But you're right, adoption is the key metric. When I was at Dubai Smart Government, we had a similar issue with the smart workspace rollout. What worked for us was identifying champions in each entity—people who would use the tool constantly and show their colleagues how it made their lives easier. Can we do something like that? Person #35: We already have a champions program. The Tomouh hub has a section for Copilot tips and tricks. But the uptake is limited. I think the problem is that a lot of the civil servants are not comfortable with AI tools yet. There's a fear that the tool will replace them or that they'll make a mistake and get in trouble. Person #34: That's a cultural issue as much as a technical one. The Undersecretary needs to send a stronger message from the top that this is about augmenting their work, not replacing them. Maybe we can have some of the senior leaders do a demo at the next leadership forum. Show them how the tool can draft a response to a minister's query in thirty seconds instead of thirty minutes. Person #35: That's a good idea. Al Jadaara cohort is starting in October. Maybe we can have a session on AI tools as part of the curriculum. I'll coordinate with the GovAcademy team. Person #34: Please do. And let's also track the usage data more carefully. I want to see a weekly report on active users per entity. If we see an entity with very low adoption, we can reach out to their leadership and understand what the blockers are. Person #35: I can have that ready by next Monday. The data is all in the telemetry system—it's just a matter of putting together the dashboard. Person #34: Great. What about the data centers? We're closing down the old ADDA facility in Mussafah. Any issues with the decommissioning? Person #35: The decommissioning is on schedule. We've moved all the critical workloads to the new sovereign cloud. The remaining workloads are non-critical—some old file shares and backup tapes that we're in the process of archiving. The facility should be fully empty by October 31st. Person #34: Are we sure we've got everything? Because once we hand back the keys, it's very difficult to go back and get anything we missed. You know the story of the entity in the US that left a server running in a closed data center for two years and only discovered it when the utility bill was forwarded to the new owner. I don't want that to be us. Person #35: We've done a thorough inventory. Three sweeps of the entire facility. The last one was last week. And we've got a checklist that every system owner has signed off on. I'm confident we haven't missed anything. Person #34: Alright. And what about the new facility? Is the expansion on track? Person #35: The expansion is on track. The construction of the new hall is complete, and we're in the process of installing the cooling systems. The target for go-live is still mid-November. The Indian vendor is handling the installation, and they've been hitting their milestones so far. Person #34: Good. We're going to need the capacity. I had a call this morning with the Ministry of Interior—they want to move their entire IT infrastructure to the sovereign cloud by the end of next year. That's a big workload. We need to make sure we have the room. Person #35: Really? The Ministry of Interior? I thought they had their own data centers. They were always very protective of their infrastructure. Even before the merger, they had their own setup and they didn't want anyone else touching it. Person #34: That was before. But I think the Chairman has had some conversations with their leadership, and they've agreed to come on board. It's a huge win for the Digital Strategy—having the Ministry of Interior as a flagship partner. But it's also a huge responsibility. If we don't deliver, it'll be a black mark on the whole program. Person #35: Well, we'll deliver. We've done it before—we brought the Department of Health onto the cloud in nine months. That was a massive migration. The Ministry of Interior is even bigger, but we have the experience and the tools now. We just need to make sure we have the resources. Person #34: That's the key. We need to start planning now, even if the migration doesn't start until next year. I want a preliminary assessment of their infrastructure—a high-level inventory of their systems, their dependencies, their data flows. We can use that to build a migration plan. Let's get a small team on that. Person #35: I can assign two people from the cloud architecture team. They'll need access to the Ministry's systems, though. That might take some time to arrange—there's a lot of bureaucracy. Person #34: I'll talk to the Undersecretary about getting the access arranged. We don't want to wait. The sooner we start, the better. At the end of the day, this is about building trust with the entities. If they see us being proactive and organized, they'll be more willing to work with us in the future. Person #35: Absolutely. Now, before I forget—there's one more thing. I got a WhatsApp message from the IT Director at the Department of Energy this morning. He's asking about the migration deadlines. Apparently, they've heard that DMT is getting an extension, and they want to know if they can delay their cutover as well. Person #34: And what did you tell him? Person #35: I told him that as far as I know, there's no extension being granted, and that he should stick to the original schedule. But he wasn't happy. He said—and I'm paraphrasing—"the news is spreading through the entity IT community, and people are getting nervous. Person #34: That's the problem with informal channels. Rumors spread faster than official announcements. We need to get ahead of this. Let's send a bulletin to all the IT directors in Wave Three, clarifying that the schedule is unchanged and that DMT's situation is being handled at the executive level. Person #35: I'll draft that today and send it to you for approval. Should we copy the Chairman's office? Person #34: No, not yet. Let's see if the bulletin is enough to calm things down. If we still get pushback, then we escalate. Person #35: Okay. So, moving on—do we have a date for the next steering committee meeting? Person #34: I think it's next Tuesday. I'll confirm with the Chairman's office. The agenda should include the Wave Three status, the Copilot adoption update, and the Ministry of Interior preliminary assessment. Anything else you want on there? Person #35: I want to talk about the budget for the new data center cooling system. The initial estimate from the vendor was lower than what we're actually seeing in the bids. We're probably going to need an additional five million dirhams. The Department of Finance is not going to like that, but we need the capacity. Person #34: Five million? That's a significant amount. What's driving the increase? Person #35: The original estimate was based on a standard cooling setup, but the new hall has higher density racks, so we need a more advanced liquid cooling system. The Indian vendor has quoted us for the upgraded system, but it's more expensive. We could go with a cheaper option, but it would limit the density we can support, and we'd have to do another upgrade in two years. Person #34: So we either pay now or pay later. At the end of the day, it's better to do it right the first time. I'll support the additional budget request. But I want you to have a detailed analysis ready—the cost breakdown, the ROI, the alternatives. We need to make the case to the Department of Finance, and we can't just say "we need more money. Person #35: I'll have the analysis ready by Thursday. I've already asked the team to put together the numbers. I think we can make a solid case. Person #34: Good. Now, is there anything else we need to discuss? I know we're running late, and I have another meeting in fifteen minutes. Person #35: One more thing, and it's quick. The Oracle support contract. We've got their renewal coming up in December. The account manager has been pushing for a meeting to discuss next year's pricing. They're expecting a significant increase—something like eight percent. I'm not sure we should just accept that. We've been a good customer, and we could push back or even look at alternative options. Person #34: What alternatives? We're pretty locked into the Oracle stack. The ERP, the database. Moving to something else would be a massive undertaking. Person #35: I know, but that doesn't mean we can't negotiate. Maybe we can get them to throw in some extra services at no cost, or extend the payment terms. Or we could threaten to move some workloads to open-source databases—SAP Hana or something like that. I know it's not realistic in the short term, but it gives us leverage. Person #34: Hmm. I'm not sure playing hardball with Oracle is the right move. They're a strategic partner, and we need them for the next five years at least. But we shouldn't just accept an eight percent increase without understanding what's driving it. Let's have a meeting with them, understand their position, and see if we can find a compromise. Person #35: I'll set that up. Maybe we can have it in the first week of September. I'll bring the procurement team along, just to show that we're serious about the budget. Person #34: Good. And if they don't budge, we can always bring it up at the Executive Council. The Chairman has a relationship with their regional CEO. A call from him might help. Person #35: That's a good card to play. Alright, I think that's everything on my list. Do you want to go through the agenda items from the last meeting? Or can we wrap up early? I know you have another meeting. Person #34: Let's wrap up. We've covered the important stuff. You'll send me the one-pager on DMT, the bulletin to the IT directors, and the budget analysis. Let me know if anything changes with the Wave Three situation. And keep me posted on the Copilot adoption numbers. Person #35: Will do. I'll send you a quick email tonight with the action items. And I'll make sure the data engineers are assigned to DMT by tomorrow morning. Person #34: Perfect. Good work today. I'll talk to you soon. Person #35: Thanks, boss. Take care. (Pause) Actually, wait—one more thing before you go. I just got a message from the SOC. They're saying there's a potential incident. Can you hold for a second? Person #34: Sure. What's happening? Person #35: Give me a second, let me read the message. (Pause) Okay, so they're flagging unusual network traffic from the Department of Municipalities and Transport. Someone inside DMT is accessing a large volume of data from the legacy ERP system. The SOC thinks it might be the vendor doing the data cleansing, but they haven't been able to confirm. Person #34: That's concerning. Is it a legitimate access or could it be something malicious? Person #35: Hard to say right now. The SOC is investigating. They're checking the source IP addresses against the vendor's known ranges. They're also looking at the volume of data being accessed—if it's a normal data export, it should be in the range of gigabytes, not terabytes. Person #34: Do we have a contact at DMT who can confirm? The IT Director? Person #35: I'll call him right now. If it's the vendor doing the cleansing, it's fine, but they should have told us in advance. If it's not the vendor, then we have a problem. Person #34: Keep me updated. I'll stay on the line for a few more minutes. This could be nothing, but we need to get to the bottom of it quickly. Person #35: Let me make the call. (Pause) I'm calling him now. (Pause) He's not picking up. Let me try his office line. (Pause) Person #34: If you can't reach him, try his deputy. Or the head of their operations center. Someone must be available on a Monday afternoon. Person #35: Yeah, I'm trying. (Pause) No answer. Let me try the deputy. (Pause) Hey, this is—yeah, I'm trying to get in touch with—no, I'm calling about—can you have someone call me back? It's urgent. Yeah. Okay, thanks. (Pause) They're going to call me back. Apparently, the IT Director is in a meeting with the Undersecretary. His phone is off. Person #34: Of course. Well, at least we know he's in the building. We can wait a few minutes. If the SOC confirms it's the vendor, then no harm done. If not, we need to escalate. Person #35: The SOC is going to send me an update in five minutes. They're doing a deeper analysis of the traffic patterns. One thing that's interesting is that the access started at 11:00 AM, which is a bit unusual for a data cleansing job. Most of those are scheduled for off-peak hours. Person #34: That is odd. Unless the vendor is working through lunch to meet their deadline. They're under pressure, and they might be cutting corners. Person #35: That's possible. But even so, they should be going through the proper channels. We have a defined process for data exports from the legacy system. If they're doing it without following the process, that's a red flag. Person #34: Let's keep an eye on it. If we confirm it's the vendor, we should still have a conversation with them about following the rules. We can't have entities and vendors doing whatever they want with data. Especially now, with the new data governance framework coming into effect. Person #35: Agreed. I'll have the SOC document everything, even if it turns out to be benign. We might need that evidence later. Person #34: Yeah. Good thinking. Let me know what they find. And if you can't reach the IT Director in the next ten minutes, we'll escalate to the Undersecretary. He should know that there's unauthorized access to his entity's systems. Person #35: I'll call you back as soon as I hear from the SOC. Unless you want to wait on the line? Person #34: I'll wait. I can cancel my other meeting. This is more important. Person #35: Appreciate it, boss. Okay, let me just... (Pause) I'm getting a notification now. Let me check. (Pause) The SOC says it's the vendor. They've confirmed the IP addresses match the vendor's known range. They're also seeing a pattern of queries that matches the data cleansing script they use. Person #34: So it's the vendor. They didn't think to inform us that they were going to be doing this during business hours? Person #35: Apparently not. I'll have a conversation with their project manager later today. This is exactly the kind of unplanned activity we've been trying to prevent. It's not malicious, but it's not disciplined either. Person #34: At the end of the day, we need to make sure this doesn't happen again. The change control process exists for a reason. If the vendor needs to do something outside of the agreed schedule, they need to go through the proper channels. Person #35: I'll make sure they understand that. And I'll also ask the SOC to tighten their monitoring of the legacy system. We should be able to detect these kinds of activities in real time, not just when someone manually reviews the logs. Person #34: Good. Alright, I think we're done here. Thanks for the thorough updates. I'll see you at the steering committee. And good luck with the DMT situation—hold the line, but be helpful. Person #35: Will do. Thanks, boss. Take care. (Pause) Oh, and I'll send you that email tonight. Person #34: Sounds good. Goodbye. Person #35: Actually, boss, wait. One thing before you go. I know we said goodbye, but we need to talk about the migration wave three. It's not looking good. Person #34: Oh, you're still there? I thought you’d already hung up. What’s the problem? Person #35: I’m still here, yeah. The issue is the Department of Energy. They’re not ready. We have the cutover scheduled for the 14th of September, and they haven't signed off on the security baseline. They’re also claiming the data cleansing isn’t finished. Person #34: The 14th? That’s two weeks away. I thought we pushed that date back from July. What happened with their IT director? I thought we had a commitment. Person #35: We did. We had a verbal commitment. But you know how they are. The new IT director—what’s his name, Al Mansoori—he’s been there for like three months, and he’s been raising questions about the cloud readiness assessment. He says the report is outdated. Person #34: Outdated? We ran that assessment in April. It was valid for six months. We told them exactly what they needed to do. Did they do any of it? Person #35: Some. They upgraded the network link, that’s done. But the data classification? Not done. They have like forty thousand unclassified files, and the data owner hasn’t even looked at them. And their legacy Exadata database? We told them to plan for the migration, but their team hasn’t started. They think they can do it in a week. Person #34: They don’t have a week. We need six weeks at least to migrate, test, and validate. And we’re not moving unclassified data. That’s a security requirement. If they can’t classify, they don’t move. Person #35: Exactly. So I need your call. Either we delay them to wave four, or we put them on a conditional waiver and let them move unclassified data later. But I don’t like waivers. That’s how we end up with you know what. Person #34: Yeah, waivers are a bad idea. Last year we gave one to the, uh, the Culture Authority, and they still have data on an old HP server in their basement. Remember that? They kept it as a souvenir. Person #35: (laughs) I remember. It was still plugged in. We had to send a team physically to unplug it. Person #34: Okay. So we don’t do waivers. We move them to wave four. We can swap in the Department of Justice? I think they're ready, right? They were the backup if anyone dropped. Person #35: DoJ is ready, yes. They've completed everything. But their data center is smaller, and they have a mainframe that we need to handle separately. If we swap them in, we might have to adjust the schedule for wave four. And the, ugh, the Department of Finance might complain. Person #34: Finance? Why would they complain? Person #35: Because DoJ’s upgrade affects the court systems. They use a joint system with, with the Public Prosecution. And there’s some shared infrastructure with the Ministry of Justice, but that’s federal. You know how these cross-emirate things get. Person #34: Yeah, cross-emirate, that’s always fun. But we can manage it. We can make a decision today. Tell DoE they’re rolling over. We need to send them a formal letter today, not tomorrow. Let's get the legal team to draft it. Person #35: I'll get Rashed on it. He’s already drafted a template for the wave delays. We have the precedent from the, you know, the first wave when the Department of Health dropped out. So it’s a quick change. Person #34: Good. And what about their legacy infrastructure? The plan was to exit their physical data center by December. If they’re delayed, that entire exit date is now at risk. Person #35: That’s the thing. Their data center contract with Etisalat ends on, hmm, let me check my notes. December 31. Yeah. If we don’t move them to the cloud by December, we have to renew. And the cost of renewal is not, it's not pretty. We negotiated a one-year extension at three million, but only if we sign by the end of next month. Person #34: Three million? We could have built a new data center for that. That’s just to rent the same racks? For one year? Person #35: Yes, and it doesn't include power. It’s, frankly, a joke. But Etisalat knows we don’t have a choice. We should have started the exit last year when we had more rooms. Person #34: Well, we didn't. So now we negotiate. Let me talk to the Undersecretary. He might be able to lean on the Etisalat side through the Executive Council. They’re not going to want to be the reason we miss our exit targets. That’s a public number. Person #35: You think he would do that? I mean, we have a good relationship with them, but they know we need them. We’re not a huge customer individually, but as the government we are. Person #34: We are the government. We’re the smart government. They can’t hold us hostage, habibi. We’ll make a call. Person #35: Okay, I’ll book that meeting with the Undersecretary for tomorrow morning if you want. Person #34: Do that. And also, I want to give him a heads-up about the vendor incident today. He might have heard already. The DMT IT Director is not going to keep quiet. He’s probably going to complain to the Inspector General about the SOC monitoring his vendor. Person #35: Wait, should we worry about that? We didn’t do anything wrong. The vendor was doing unplanned activity. We spotted it, we confirmed the pattern. We didn't block them. Person #34: I know, I know. But you know how these things get. A complaint about "excessive surveillance" will become a story. They might say we're spying on entities. We need to be proactive. Let’s prepare a one-page summary and send it to the Undersecretary’s office before the end of today. Include the IP addresses, the query pattern, and the fact that the vendor confirmed it. Person #35: Okay, I'll have the SOC compile it. They’re already writing an incident report. I can tell them to make it a summary version. Person #34: Perfect. And include a timeline, and note that we tried to contact the entity’s IT director before taking any proactive measures. I mean, we didn't take any, we just observed, but we tried to contact him. Person #35: We did call him. Three times. No answer. Person #34: Good. Promise me you'll preserve those call logs. Person #35: Already will. The SOC logs everything. We have the call timestamps. Person #34: Fine. Okay, anything else on the migration? Person #35: Yes, one more thing. The DMT migration—we did that last month, right? And we saw an uptick in batch job failures on their SAP system after we moved to Azure. We’ve been telling them that the issue is not the cloud but their middleware. But they keep blaming us. And now they want to use that as a reason to stop their next workload migration. Person #34: The DMT again? They're always a pain. Did we check the logs? Is it actually us? Person #35: The logs show a misconfiguration in their IDoc processing, it has nothing to do with the network. We even did a same-region, same-size test in our sandbox and it worked perfectly. The problem is their version of SAP and their custom code. They haven't updated it in, I don't know, five years. Person #34: Ah, the classic "we can't upgrade because the license is too expensive" story. And now they want to delay the rest of the migration? We have a meeting later this week with their deputy secretary. Are you coming? Person #35: Yes, I'm scheduled for Thursday, the 10th, 10am. Person #34: Good. Bring the evidence. We need to be firm. We'll offer to send a technical team to help them re-align their middleware. But we must not accept the blame. I know the media likes the "DGE delays government entity project" headline, but we can't allow that. Person #35: Yeah, it's always us. They don't mention the 53 entities that have migrated successfully. Person #34: Let's change the subject. How is the TAMM situation? I saw some critical comments on social media about the fee hallucination last week. Or was it the X platform? I saw a screenshot. Person #35: Oh, that. Yeah, the AI assistant told someone that they'd need to pay a "knowledge fee" to renew a business license. There is no such fee. It hallucinated. The press picked it up. CX team was furious. Person #34: I saw the headline: "TAMM Assistant invents new government fees." That was embarrassing. What's the fix? Person #35: We've temp disabled the fee-related intents while the AI team retrains the model. They're adding new guardrails. But more importantly, we're realizing that the integration between the AI assistant and the payment gateway is too loose. The assistant can access the fee catalog, but it also has access to some old, unverified data. The vendor who built the fee catalog left, and no one maintains it. Person #34: That's an operational gap. We need a data owner. Whose responsibility is the fee catalog? Person #35: Technically it's the Department of Finance, since they fix the fees. But the catalog on the platform is managed by the CX team, not by us. They don't have the expertise. Person #34: So we need a governance board just for the fee catalog. That's insane. But let's not fix it today. Let's make a note for the strategy steering. I want to raise it at the next meeting with the Undersecretary. We'll ask him to mandate that DOF appoints an owner for the fee catalog and that we have a weekly sync. Person #35: That's a good idea. I'll include it in the summary for the meeting tomorrow. Person #34: Now, what about the cyber incident? I mean the SOC noticed some other suspicious activity during the same time. The past week, we've had a few brute force attempts on the remote access VPN, any concerns? Person #35: Nothing that we haven't seen before. The SOC has automated blocking. We had 1,400 attempts last month, but zero successful. The public IPs were from a few different countries, but we’re used to that. It's noise. Person #34: Keep monitoring. We don't want a repeat of the incident at the Department of Transport, back in 2023. That was a nightmare. Person #35: Yes, I remember that. It was a major breach. They had to pay. Person #34: Actually, they didn't pay. The government didn't pay. We refused. But it was a mess. Anyway. Person #35: Okay, shifting to the Copilot rollout. The Microsoft Team is asking about the next batch of licenses. We have 5,000 more licenses to turn on by the end of September. Are we on track? Person #34: The procurement is approved. We're just waiting for the security review to complete. The Internal Audit team is concerned about how Copilot might process sensitive data. We’re working through that. Person #35: But we've already done an assessment for the first batch, right? The security baseline was approved. Why are they re-reviewing? Person #34: Because some entity, I think it's the Department of Culture, complained that their staff members were using Copilot to summarize confidential memos, and they said they didn't know. Actually that's not true. We sent a note to all entities saying that Copilot can access the user's permissions. But the legal team wants us to confirm that every user must have proper data classification before they turn it on. Person #35: That's a lot of work. We'd have to go entity by entity. And we have 35,000 users, even if we turn on licenses, we can't manually train each one. Person #34: We should not. We’ll automate the training with a short video and require them to click through a small quiz. The licensing process will be conditional on the head of entity signing a letter that they have completed a classification review. Person #35: That sounds like a lot of manual work for the HR teams in each entity. Person #34: I know. But we don't want a scandal. The media could easily say "Government AI is spying on your data." We must be careful. You remember the "Copilot nonsense" in the press last month. Person #35: Yes, I remember. The article was badly misinterpreted. But okay, I’ll wait for the legal guidance. Do we have a timeline? Person #34: They promised to finalize by Sunday, the 6th. So we have a week. Then we can start the activation on the 8th. Person #35: Sunday is the 6th? Wait, the 6th is a Sunday, yes. Good. Person #34: Now, what about the data center exit for the legacy government data center in Al Ain? We had a team there last month. What's the status? Person #35: They're still doing a physical inventory, but they found a bunch of equipment that wasn't on the list. Some old routers, a few boxes of tapes, and a switch that's still connected to a production network. We told them to disconnect it, but they said they don't know what it does. Person #34: That switch? Is it still connected? Person #35: As of last Thursday, yes. The team is scared to disconnect it because they don't know if it's supporting something. The vendor who set it up retired. Person #34: That's a disaster waiting to happen. Listen, we should schedule a maintenance window in the next couple of weeks. Let's disconnect the switch, and if something breaks, we'll answer questions later. Person #35: I'll talk to the operations team. We can schedule it for the 10th or 11th, but we have to coordinate with the telecom providers. It might be a small window. Person #34: Do it. And while we're at it, check whether we have any other undocumented devices in the data center. This is the last one, right? Person #35: There's a rack of old Sun servers in the corner, they've been decommissioned, but they're still there. We're waiting for the asset disposal vendor to pick them up, but they're not responding to our emails. Person #34: Are they a government-approved vendor? Are they on the procurement list? Person #35: Yes, they won the contract for the e-waste disposal earlier this year. But they've been, you know, slow. I'll chase them again. Person #34: Good. Now, what about the OpenAI and G42 things? I saw the announcement of a new sovereign AI cloud region that they're setting up in Abu Dhabi. But we're supposed to be the ones doing that. Are we involved? Person #35: Not directly. That's more the main AI Office, not our department. But we’re coordinating with them on the cybersecurity framework. They're using the same Microsoft stack, with some adaptation. Person #34: I hope they talk to us before they deploy their first workload. We cannot have two sovereign clouds with different standards. That would be a nightmare. Person #35: We have a joint working group, but they only meet monthly. The next meeting is scheduled for the 16th of September. I'll raise it. Person #34: Make it a priority. We need to establish a common set of standards. Otherwise, we'll have multiple isolated clouds. Person #35: I'll put that on the agenda for the DG meeting as well. Person #34: Let's talk about the strategy. We have the 2025-2027 digital strategy. We are supposed to have 200 AI use cases. How many have we actually deployed? Person #35: As of last week, we have 37 in production, 44 in pilot, and about 60 in development. The rest are just ideas. Person #34: Only 37? That's, um, that's 18%. We're halfway through the strategy timeline, and we're only at 18%? We're far behind. Person #35: Yes, but many of those use cases are larger. Some are entity-wide. The administrative burden is significant. Every pilot requires infrastructure, readiness, and ethics review. Plus there's the procurement. Person #34: I understand, but we need to accelerate. I want to see 50 in production by the end of this year. That's an ambitious target, but we can do it. We need to prioritize quick wins. Identify the top 10 use cases that can be delivered in 90 days. We'll make a special cell for them and give them a fast-track. Person #35: A special cell? Like a tiger team? That usually creates conflicts with the regular teams. Person #34: It's about focus, not conflict. We'll assign dedicated resources, and the team will report directly to me. No other project can pull people away. We did the same thing in Dubai when we launched the "smart Dubai" app. It worked. Person #35: Boss, sometimes the "Dubai way" doesn't map exactly to Abu Dhabi. But we can try. I'll create a spreadsheet and share it with you by Friday. Person #34: Not Friday. Thursday. We need to act fast. Person #35: Fine, Thursday evening. Person #34: Now, there was something else. The World Governments Summit is in February. We have to prepare the DGE exhibition. Do you know what we're doing? Person #35: I think the communications team is drafting a concept. Something about "AI-native government." But I haven't seen the details. It's still in the early planning. Person #34: I want us to have a live demo of a cross-entity journey. Maybe something to do with business setup. Show a new business license being approved with AI assistance in 15 minutes. The demo must be on TAMM and the back-end must be real, not a mock. Person #35: That's going to upset the CX team because they'll have to do a lot of integration. But I can get a few engineers on it. Person #34: Don't worry about the CX team. I'll talk to them. Let's give them a challenge. They'll deliver something great. Person #35: Okay. And also, for the WGS, there will be a delegation from Kazakhstan. They're interested in the digital strategy. We might need to prepare a briefing pack. Person #34: Yes. Noted. We can do a short, one-page overview. We'll get the governance directorate to prepare it. What about the WGS side events? Are we hosting a roundtable? Person #35: I think we have the approval for a roundtable on "Future of Government Service." I'll check with the Events team. Person #34: Fine. Now, we should also talk about the staffing situation. I heard we've lost two cloud engineers to a bank recently. Are we having retention problems? Person #35: Unfortunately, yes. The bank, I think it's the new ADCB digital branch, they're offering higher salaries. And the private sector is hungry for people with cloud skills. Our cloud engineers are skilled, they are certified, and they’re getting offers. Person #34: We need to approach HR about the pay scale. Maybe we can offer variable pay or a sign-in bonus. But that’s not just for my sector; it's a government-wide issue. We need to see what the new HR law allows. It's supposed to be more flexible. Person #35: The new HR law, 2026, it does have a provision for hard-to-fill roles. We can propose some of our roles to be classified as such. Then we can set a higher salary ceiling. But the process goes through the Classification Committee. It takes months. Person #34: Let's start the process. It's better than losing people. We have the Abu Dhabi School of Government too, we can offer educational opportunities. But no, we don't need to discuss that now. Let's note it down. Person #35: Noted. Person #34: Okay, we're getting long here. Let's wrap up soon. But first, what's the update on the Microsoft Azure capacity? Are we close to consuming the full datacenter capacity? We have a plan for three more regions in the country. Person #35: The first region is at 60% consumption. We have the second region coming online in March 2027. The third is still in design. The problem is power supply. The national grid can't handle another data center at the Al Ain location. We're looking at other sites. Person #34: Have you looked at Masdar? They have extra solar capacity. Person #35: We have a preliminary agreement with Masdar. But they want us to sign a long-term energy purchase agreement, and the salaries of the finance team are not positive. I mean, they're not sure about the cost. We'll need to bring Finance into the loop. Person #34: Okay, that's a normal step. Let’s set up a meeting with them next week. I think they'll be supportive because it's a strategic initiative. Person #35: Alright. Also, the oracle fusion ERP rollout to the last group of entities. The go-live is scheduled for November. The Department of Community Development is the last one. They had some delays in their financial data mapping. The consultant said they need two more weeks. Should we allow a delay? Person #34: No. The go-live date is set. If they delay, it will affect the budget. We cannot have a partial rollout. We'll put their finance director in contact with the DOF to help. But if they still can't meet it, we have to escalate to the Secretary. I don't want to do that. Person #35: Let me check if the extra support from the Deloitte team might be helpful. They're already on site. Person #34: Use them. They've been doing the heavy lifting for the integration. Person #35: Yes. Person #34: Anything else on the technical side? This cloud thing is, what else? Person #35: Just one more thing, the new service portal for IT support. ServiceNow's implementation is going well, we've trained about 20 support agents last week. But there's a backlog of tickets from the transition period. We have about 2,300 tickets that are still unassigned. The service desk is overloaded. Person #34: That's not a technological issue. It's an operational issue. We should be stricter about the SLA, and if the vendor can't handle the volume, we need to increase their resources. Person #35: We can ask them to bring on more staff. They've been responsive. I'll give them a call today. Person #34: Do it. Now, let's think about the change freeze for the government during the National Day holidays. We have the long weekend in December. When should we announce the freeze? Person #35: I was thinking the last week of November, but we need to freeze all changes from December 1 to December 5, maybe extend to December 7 to be safe. That's a good time because many people will be out. Person #34: I agree. We'll need to coordinate with the entities, and the Central Bank if they are open. They don't have the same holidays. We'll send a circular by mid-November. Make a note. Person #35: Will do. Person #34: Now, what about the Abu Dhabi government procurement gate? The digital team is implementing the new module for IT orders. Is that on track? Person #35: That’s actually with Corporate Services, but they mentioned to me that the e-guide version has been delayed. There is a dispute with the vendor about the user interface. I'm not concerned, but we may need to delay the go-live to October, after GITEX. Person #34: That might be fine. We don't want to launch something broken at GITEX. We have better things to show. Person #35: Agreed. Person #34: I'm also curious about the Customer Experience team. They are doing some "mystery shopping" on the integrated services, like the "birth to lifelong" journey. Any issues they found that impact the cloud infrastructure? Person #35: No, not really. They have some performance issues with the portal when taking a video call. It's the video platform's bandwidth on certain connections. We may need to enable a fallback to standard voice calls. Person #34: Good. That’s considerate. Let’s not forget to extend that to the "tall buildings" in certain areas where internet might be slow. Person #35: We'll include it in the next service pack. Person #34: Now, we touched on the DMT. That's a sensitive one. Is there anything else regarding DMT? I know they had an issue with their employees' payroll file last week. Was that resolved? Person #35: Yes, the file was corrupted, but we restored from backup. They lost a day of data. The issue was due to an external system. I think their HR department tried to upload an Excel file with wrong formatting. It's not our problem, but we helped. Person #34: That's very kind of us. They should be grateful. Person #35: I wouldn't call them grateful, but they thanked us. That's enough. Person #34: Let's also talk about the "open data" initiative. We're supposed to release more datasets to the public. Have we made any progress? Person #35: We have the portal updated, but we only have 300 datasets. The target is 1,000 by the end of 2026. Many entities are reluctant to share data. They're worried about privacy and misinterpretation. Person #34: We'll need an Executive Council mandate to force entities to identify non-sensitive data. Maybe we can add that to the social agenda for the next cabinet meeting. But it will take time. For now, we can work with the two or three large entities like the Health, Education, and Transportation. Person #35: That's a good idea. I'll get the data office to draft a list of the low-hanging fruit. Person #34: Good. Now, um, I think we've covered a lot. Is there any critical incident I haven't heard about? Person #35: We had a small incident at the Mawaheb platform on Saturday. A certificate generation system broke, causing some courses to not issue certificates. We fixed it in two hours. It was due to a database lock. No data loss. Person #34: Mawaheb? That's not your area though. But okay. Good to hear they fixed it. Person #35: We were on call because they use our cloud. It was acceptable. Person #34: All right. I think we should conclude. We have a lot of action items. I will send you a brief email summarizing the decisions we made today. Circulate it to your team. Person #35: Yes, please. I'll need that for the record. Also, the DMT meeting on Thursday. I'm still expecting you to lead it. Bring the evidence. Person #34: I'll be there. I already have the email. It's on my calendar. I'll bring the battle axe. Person #35: (laughs) See you on Thursday. And I'll send you the incident report before the end of today. Person #34: Perfect. Good luck with the Etisalat meeting. Person #35: Thanks, boss. I think I'll need it. Person #34: I'll see you soon. Goodbye. Person #35: Goodbye. Person #34: Wait, one more thing. I forgot to ask. Did you receive the revised budget figures from the Department of Finance for next year? For the cloud services? Person #35: Um, yes, we did. They approved the same as our initial request. No cuts. But they want us to submit a list of any additional software subscriptions. They need it by the 15th of September. Person #34: Okay. That's a deadline we have to make. Add it to your list. Person #35: It's already in my list. I'll have the technical team update the subscriptions by next week. Person #34: Good. That's all. Goodbye. Person #35: Goodbye, boss. Person #34: Okay. Take care. (End of the call)