TOTVS iPaaS - Notice history

Experiencing partial outage

Serviço de Autenticação - TOTVS Identity - Operational

100% - uptime
01 Apr 2025 · 100.0%01 May · 100.0%30 Jun · 100.0%

Previous page

Next page

01 Apr 2025
01 May 2025
30 Jun 2025

Aplicação - Operational

100% - uptime
01 Apr 2025 · 100.0%01 May · 100.0%30 Jun · 99.88%

Previous page

Next page

01 Apr 2025
01 May 2025
30 Jun 2025

Serviço de Nuvem - GCP - Major outage

100% - uptime
01 Apr 2025 · 100.0%01 May · 100.0%30 Jun · 100.0%

Previous page

Next page

01 Apr 2025
01 May 2025
30 Jun 2025

Rastreabilidade - Operational

100% - uptime
01 Apr 2025 · 100.0%01 May · 100.0%30 Jun · 100.0%

Previous page

Next page

01 Apr 2025
01 May 2025
30 Jun 2025

Monitoramento - Operational

100% - uptime
01 Apr 2025 · 100.0%01 May · 100.0%30 Jun · 100.0%

Previous page

Next page

01 Apr 2025
01 May 2025
30 Jun 2025

Third Party: CloudAMQP → AWS → AWS ec2-us-east-2 - Operational

Third Party: Google Cloud Platform → Google Cloud Networking - Operational

Third Party: Google Cloud Platform → Google Cloud SQL - Major outage

Third Party: Google Cloud Platform → Google Cloud Storage - Major outage

Third Party: Google Cloud Platform → Google Cloud Storage - Major outage

Third Party: Google Cloud Platform → Google Compute Engine - Operational

Third Party: Elastic Cloud → GCP Iowa (us-central1) → Deployment hosts: GCP us-central1 - Operational

Third Party: Elastic Cloud → GCP Iowa (us-central1) → Deployment metrics: GCP us-central1 - Operational

Third Party: Elastic Cloud → GCP Iowa (us-central1) → Elasticsearch connectivity: GCP us-central1 - Operational

Third Party: Cloudflare → Cloudflare Sites and Services → DNS Firewall - Operational

Third Party: Cloudflare → Cloudflare Sites and Services → Firewall - Operational

Third Party: Cloudflare → Cloudflare Sites and Services → Gateway - Operational

Notice history

01 Jun 2025

Instabilidade no TOTVS iPaaS
  • Postmortem
    Postmortem

    🛑 Resumo do Incidente – Google Cloud e Workspace (12 de junho de 2025)

    Duração: 3 horas

    Regiões afetadas: Global (com recuperação mais lenta em us-central1, utilizada pelo TOTVS iPaaS)

    Serviços afetados: Diversos produtos do Google Cloud, incluindo APIs, interfaces de usuário, BigQuery, Compute Engine, Cloud Storage, entre outros.

    📌 O que aconteceu:

    Uma atualização automática de cotas inválida na GCP foi aplicada ao sistema de gerenciamento de APIs, gerando erros 503 nas chamadas externas e interrupções em serviços. A maioria das regiões se recuperou em até 2 horas, mas us-central1 teve sobrecarga e demorou mais.

    💥 Impacto aos clientes:

    • APIs e interfaces intermitentes

    • Interrupção do serviço do TOTVS iPaaS

  • Resolved
    Resolved

    Todos os serviços utilizados no TOTVS iPaaS foram reestabelecidos.

    Para acompanhar o incidente da GCP acesse: https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW

  • Update
    Update

    O serviço de database e storage foi reestabelecido, ainda com bastante lentidão.

    Estamos monitorando a plataforma, aonde já é possível logar.

  • Monitoring
    Monitoring

    Os engenheiros da google identificaram a causa raiz e aplicaram as devidas medidas de mitigação.

    Embora os engenheiros tenham confirmado que a dependência subjacente foi recuperada em todas as localidades, exceto em us-central1 (O que usamos no TOTVS iPaaS primáriamente), ainda estamos enfrentando diferentes níveis de impacto em produtos individuais do Google Cloud. Todas as equipes de engenharia responsáveis estão ativamente envolvidas e trabalhando na recuperação dos serviços.

    Ainda não temos uma estimativa para a recuperação completa dos serviços.

  • Update
    Update

    Resumo: Vários produtos do GCP estão enfrentando problemas de serviço.

    Descrição: Estamos enfrentando problemas de serviço com vários produtos do GCP desde quinta-feira, 12 de junho de 2025, às 15:20

    Equipe do google continua investigando o problema. Pedimos desculpas a todos os afetados pela interrupção.

    Sintomas: Vários produtos do GCP estão apresentando diferentes níveis de impacto nos serviços.

    Solução alternativa: Nenhuma no momento.

  • Identified
    Identified

    Estamos enfrentando uma interrupção de conexão com os serviços da Google Cloud Platform (GCP).

    No momento, todos os nossos serviços hospedados na GCP estão indisponíveis. Nossa equipe está em contato com o provedor para identificar a causa e restabelecer a conectividade o mais rápido possível.

    Agradecemos a compreensão e manteremos novas atualizações por este canal.

  • Investigating
    Investigating
    We are currently investigating this incident.
[TOTVS iPaaS] Production Update
  • Completed
    11 June 2025 at 11:00 PM
    Completed
    11 June 2025 at 11:00 PM
    Maintenance has completed successfully
  • In progress
    11 June 2025 at 9:00 PM
    In progress
    11 June 2025 at 9:00 PM
    Maintenance is now in progress
  • Planned
    11 June 2025 at 9:00 PM
    Planned
    11 June 2025 at 9:00 PM

    Dear customer,

    We inform you that we will perform a routine update on TOTVS iPaaS with the aim of improving the performance, security and general user experience of the product.

    During this period, access to the system may be temporarily unavailable.

    We appreciate your understanding and are available for any questions or support needed.

    Best regards, TOTVS Team.

[TOTVS iPaaS] Production Update
  • Completed
    04 June 2025 at 11:00 PM
    Completed
    04 June 2025 at 11:00 PM
    Maintenance has completed successfully
  • In progress
    04 June 2025 at 9:00 PM
    In progress
    04 June 2025 at 9:00 PM
    Maintenance is now in progress
  • Planned
    04 June 2025 at 9:00 PM
    Planned
    04 June 2025 at 9:00 PM

    Dear customer,

    We inform you that we will perform a routine update on TOTVS iPaaS with the aim of improving the performance, security and general user experience of the product.

    During this period, access to the system may be temporarily unavailable.

    We appreciate your understanding and are available for any questions or support needed.

    Best regards, TOTVS Team.

01 May 2025

[TOTVS iPaaS] Production Update
  • Completed
    27 May 2025 at 11:00 PM
    Completed
    27 May 2025 at 11:00 PM
    Maintenance has completed successfully
  • In progress
    27 May 2025 at 9:00 PM
    In progress
    27 May 2025 at 9:00 PM
    Maintenance is now in progress
  • Planned
    27 May 2025 at 9:00 PM
    Planned
    27 May 2025 at 9:00 PM

    Dear customer,

    We inform you that we will perform a routine update on TOTVS iPaaS with the aim of improving the performance, security and general user experience of the product.

    During this period, access to the system may be temporarily unavailable.

    We appreciate your understanding and are available for any questions or support needed.

    Best regards, TOTVS Team.

[TOTVS iPaaS] Production Update
  • Completed
    21 May 2025 at 11:00 PM
    Completed
    21 May 2025 at 11:00 PM
    Maintenance has completed successfully
  • In progress
    21 May 2025 at 9:00 PM
    In progress
    21 May 2025 at 9:00 PM
    Maintenance is now in progress
  • Planned
    21 May 2025 at 9:00 PM
    Planned
    21 May 2025 at 9:00 PM

    Dear customer,

    We inform you that we will perform a routine update on TOTVS iPaaS with the aim of improving the performance, security and general user experience of the product.

    During this period, access to the system may be temporarily unavailable.

    We appreciate your understanding and are available for any questions or support needed.

    Best regards, TOTVS Team.

[TOTVS iPaaS] Production Update
  • In progress
    14 May 2025 at 1:00 AM
    In progress
    14 May 2025 at 1:00 AM
    Maintenance is now in progress.
  • Completed
    14 May 2025 at 1:00 AM
    Completed
    14 May 2025 at 1:00 AM
    Maintenance has completed successfully
  • In progress
    13 May 2025 at 9:00 PM
    In progress
    13 May 2025 at 9:00 PM
    Maintenance is now in progress
  • Planned
    13 May 2025 at 9:00 PM
    Planned
    13 May 2025 at 9:00 PM

    Dear customer,

    We inform you that we will perform a routine update on TOTVS iPaaS with the aim of improving the performance, security and general user experience of the product.

    During this period, access to the system may be temporarily unavailable.

    We appreciate your understanding and are available for any questions or support needed.

    Best regards, TOTVS Team.

01 Apr 2025

[TOTVS iPaaS] Production Update
  • Completed
    29 April 2025 at 12:00 PM
    Completed
    29 April 2025 at 12:00 PM
    Maintenance has completed successfully
  • In progress
    29 April 2025 at 10:00 AM
    In progress
    29 April 2025 at 10:00 AM
    Maintenance is now in progress
  • Planned
    29 April 2025 at 10:00 AM
    Planned
    29 April 2025 at 10:00 AM

    Dear customer,

    We inform you that we will perform a routine update on TOTVS iPaaS with the aim of improving the performance, security and general user experience of the product.

    During this period, access to the system may be temporarily unavailable.

    We appreciate your understanding and are available for any questions or support needed.

    Best regards, TOTVS Team.

[TOTVS iPaaS] Production Update
  • Completed
    15 April 2025 at 9:00 PM
    Completed
    15 April 2025 at 9:00 PM
    Maintenance is now in progress
  • Planned
    15 April 2025 at 9:00 PM
    Planned
    15 April 2025 at 9:00 PM

    Dear customer,

    We inform you that we will perform a routine update on TOTVS iPaaS with the aim of improving the performance, security and general user experience of the product.

    During this period, access to the system may be temporarily unavailable.

    We appreciate your understanding and are available for any questions or support needed.

    Best regards, TOTVS Team.

[TOTVS iPaaS] Production Update
  • Completed
    15 April 2025 at 11:00 PM
    Completed
    15 April 2025 at 11:00 PM
    Maintenance has completed successfully
  • In progress
    15 April 2025 at 9:00 PM
    In progress
    15 April 2025 at 9:00 PM
    Maintenance is now in progress
  • Planned
    15 April 2025 at 9:00 PM
    Planned
    15 April 2025 at 9:00 PM

    Dear customer,

    We inform you that we will perform a routine update on TOTVS iPaaS with the aim of improving the performance, security and general user experience of the product.

    During this period, access to the system may be temporarily unavailable.

    We appreciate your understanding and are available for any questions or support needed.

    Best regards, TOTVS Team.

[TOTVS iPaaS] Production Update
  • Completed
    13 April 2025 at 5:00 AM
    Completed
    13 April 2025 at 5:00 AM
    Maintenance has completed successfully
  • In progress
    13 April 2025 at 2:00 AM
    In progress
    13 April 2025 at 2:00 AM
    Maintenance is now in progress
  • Planned
    13 April 2025 at 2:00 AM
    Planned
    13 April 2025 at 2:00 AM

    Dear customer,

    We inform you that we will perform a routine update on TOTVS iPaaS with the aim of improving the performance, security and general user experience of the product.

    During this period, access to the system may be temporarily unavailable.

    We appreciate your understanding and are available for any questions or support needed.

    Best regards, TOTVS Team.

01 Apr 2025 to 01 Jun 2025

Next